Finally getting around to posting our materials from the talk that Chris Wysopal and I gave at BlackHat this year entitled "Static Detection of Application Backdoors." Here are the slide deck and the accompanying whitepaper:
Also, as a proof-of-concept, we had demonstrated using IDA Pro's scripting framework to detect one of the backdoor examples that we discussed -- suspicious cryptographic API calls. Specifically, it flags calls to known encryption, decryption, and/or key management functions where a constant value is passed to a specific argument position. This can help identify situations such as an application encrypting data with a hard-coded key. We had numerous requests to post the code, so here it is:
CA Veracode's binary analysis technology uses similar (but more sophisticated) techniques. We build our own intermediate representation of the binary's data flows, control flows, and range propagation which is not based on IDA Pro. We then scan that representation for backdoors in ways similar to the cryptoconst script. However, at BlackHat you're not allowed to promote your own products/services, so it wasn't appropriate for us to use it for demonstration purposes.