/jun 18, 2019

Application Security Beyond Static Analysis

By Suzanne Ciccone

There is no application security “silver bullet” – it takes a combination of testing types to effectively reduce your risk. Each testing method has a different role to play and works best when used in harmony with others.

For instance, our research showed that there are significant differences in the types of vulnerabilities you discover dynamically at runtime compared to those you’ll find when doing static testing in a non-runtime environment. In fact, two of the top five vulnerability categories we found during dynamic testing weren’t even among the top five found by static, with one not found by static at all.

Add to this the fact that applications are increasingly “assembled” from open source components, rather than developed from scratch, which means software composition analysis is an important part of your testing mix. Neglecting to assess and keep track of the open source components you are using would leave a large portion of your code exposed and leave you open to attack. 

And finally, automation alone is not enough to ensure an application is thoroughly tested from a security perspective. Some flaws, such as CSRF (Cross-Site Request Forgery) and business logic vulnerabilities, require a human to be in the loop to exploit and verify the vulnerability. Only manual penetration testing can provide positive identification and manual validation of these vulnerabilities.

Here's an overview of the different types of vulnerabilities found by different testing types:

capabilities static analysis software composition analysis dynamic analysis manual penetration testing
Flaws in custom web apps (CWEs) X   X X
Flaws in custom non-web apps (CWEs) X     X
Flaws in custom mobile apps (CWEs) X     X
Known vulnerabilities in open source components (CVEs)   X   X(1)
Behavioral issues (CWEs) X(2)     X
Configuration errors (CWEs)     X X
Business logic flaws (CWEs)       X
Repeatable process for automation X X X  
Scalable to all corporate applications X X X  
Scan speed Seconds to hours Seconds to minutes Hours Days to weeks
Cost per scan $ $ $ $$

1Penetration testing can find known vulnerabilities in open source components, but this may not be as rigorous as Veracode Software Composition Analysis, which not only systematically flags CVEs but also crawls commit histories and bug tracking tickets in open source projects to identify silent fixes of security issues.
2This is not true for all static analyzers. Veracode can exercise the code and manipulate the UI for behavioral analysis in mobile applications.

Here’s a summary of when to use each testing type:

assessment type advantages limitations
Static analysis (with entire application in scope)
  • Very broad coverage of flaw types (CWEs)
  • Looks at the flaws in the context of the entire application, analyzing all the data paths
  • Can scan any type of application, including web, mobile, desktop, or microservices
  • Scanning frequency should be in line with how often developers can review scan results
  • Use static analysis as part of Continuous Delivery pipeline and file security issues in bug tracking system
  • Can track flaw history: new, open, fixed. Important for trending reports on mean time to remediation.
  • Suitable for compliance purposes
  • Does not provide instant feedback to developers as they’re coding
  • Cannot find CWEs related to server configurations
  • Limited to code that developers can remediate.
  • Does not report vulnerabilities in third-party components (see: SCA).

Static analysis (on file level, e.g., Veracode Static Analysis IDE Scan)

  • Recommended for development teams who want to shift left in application security testing by scanning early and often. Scans usually complete in seconds.
  • Best suited when scanning multiple times per day
  • Recommended for use by developers working on the new code for continuous flaw feedback and remediation guidance
  • Developer friendliness: enhances learning, allows developers to find and address issues without exposing flaws in reports
  • Scans web applications without having to integrate with the SDLC
  • Ability to scan in pre-production and production
  • Suitable for compliance purposes
  • Scans individual files, so can only detect vulnerabilities where source and sink are in same file
  • Typically not suited for compliance scanning because scope limitations may cause false negatives
  • Does not report vulnerabilities in third-party components
Dynamic analysis
  • Scans web applications without having to integrate with the SDLC
  • Ability to scan in pre-production and production
  • Suitable for compliance purposes
Scan times are often between 12 and 24 hours for complex applications, so recommended for overnight scans, or for asynchronous scanning

Software composition analysis

 

  • Finds vulnerabilities in third-party components
  • Scans take seconds or minutes Can scan any type of application, including web, mobile, desktop, or microservices
  • Suitable for compliance purposes
Does not find flaws in first-party code

 

For more details, check out our new guide, Application Security Best Practices.

Related Posts

By Suzanne Ciccone

Suzanne is part of the content team at Veracode, working to create resources that shed light on AppSec problems and solutions.