Skip to main content
May 23, 2016

Bad Things Happen When You Don’t Measure Your AppSec Program

If you’re going to spend time, money and effort implementing an application security program, don’t lose your progress by neglecting to collect and share metrics. With strong metrics, you not only prove that your program is making a positive impact, but also identify where and how it’s working – or not working. What happens if you don’t measure? Bad things like these:

1 - You don’t get money

AppSec is far from a slam dunk when it comes to C-level support. Without clear metrics, you won’t get the C-level backing, or funds, you need to scale your program over time to cover your entire application landscape.

It’s not that corporate leaders are unaware of application-layer risk. A 2015 Veracode study found that 66 percent of corporate directors are less than "confident" about their company's ability to thwart cyberattacks, and only 4 percent are "very confident.” The problem is that most business leaders are unwilling to commit money and resources to scaling application security – unless, of course, a breach occurs. At that point, the purse strings open, but that is also when the costs skyrocket. Remarkably, one in five companies only discuss cybersecurity and risks immediately following an internal incident or an event within the same industry.

This trend reveals that security professionals need to do a better job illustrating the benefits and ROI of expanding an AppSec program. When making this case, metrics are critical. The C-suite should understand how your program is affecting the bottom line, even when nothing dramatic is occurring.

Make your case to the C-suite by collecting metrics that answer questions like these:

  • How has the application security program improved security risk posture overall?
  • How are you helping to improve the availability and operational risk of critical systems and services?
  • How are you helping to reduce time-to-delivery for important programs or cost of operations for existing services?
  • What serious risks remain to critical operations and key products or programs?
  • How do these risks translate to threats to key customers and company reputation?
  • What are you doing to systemically reduce these risks?

The bottom line is to illustrate how AppSec is part of larger initiatives, programs and issues that are important to the business. Understand what is most important to your C-level audience: moving and innovating faster? Saving costs and increasing efficiency? What customers or programs are critical to success? Tie your application security programs to the answers to these questions.

2 - You spend your money on the wrong things

Without detailed information about the state of your application security, you can’t focus your efforts, and funds, on the right areas, and your program will not be effective.

Most people will track the raw technical data created by their AppSec programs – including metrics such as number of vulnerabilities found, flaw density, flaw severity, flaw type and discovery method. But this data isn’t worth much until you use it to create operational metrics – for example, which systems have the most flaws, which flaws are seen most often and which testing methods are working best. Analytics dashboards from enterprise scanning tools and service providers are good places to derive a lot of this information.

With operational metrics, you’ll see patterns and be able to identify weaknesses and strengths in your application security program, to understand risks and trends, and to help make decisions and improvements.

These are the types of questions your operational metrics should answer:

  • Are you seeing certain kinds of vulnerabilities increase or decrease over time? Are you seeing new risks or new types of vulnerabilities? 
  • What do you need to do to improve these trends? Do developers need more training? Do you need better, faster tools for earlier testing and feedback?
  • Which systems have the highest number of vulnerabilities – or the most serious vulnerabilities and why?
  • What are the most common types of vulnerabilities being found?
  • Where are these vulnerabilities? In-house code, third party code or open source components?
  • How are vulnerabilities being found? And when?

Without spinning up your technical metrics to answer these types of questions, you are most likely not only spending money and using resources inefficiently, but also leaving applications exposed.

3 - You get breached

Without support, funds or the proper focus, your AppSec program will not be effective – leaving you at risk of a breach. For example, if you’re unaware that your third-party apps are riddled with vulnerabilities, you’re at risk. Likewise, if you don’t get the funds to scale your program to cover third-party apps, you’re also at risk. And with the Verizon Data Breach Investigations Report reporting that web application attacks are one of the most frequent patterns confirmed in breaches, this risk is very high.

Bottom line: You need to put the time and effort into gathering solid metrics about your program. Start with our new guide, Using Metrics to Manage Your Application Security Program, which is filled with detailed information on critical AppSec metrics and how to get them.

Suzanne is part of the content team at Veracode, working to create resources that shed light on AppSec problems and solutions. 

Love to learn about Application Security?

Get all the latest news, tips and articles delivered right to your inbox.