Skip to main content
May 18, 2016

In AppSec, What You Measure Is as Important as What You Do

If you’ve ever wrapped a gift and ended up with a big stripe of the box showing down the middle, you know “measure twice, cut once” is a popular saying for a reason. The need to give equal attention to measuring and doing holds true for a plethora of activities and industries, and application security (AppSec) is no exception. You can implement all the latest and greatest AppSec tools, technologies or programs, but if you aren’t measuring your efforts by tracking, analyzing and distributing metrics, you will end up with an incomplete and ineffective program – the security equivalent of that sad, half-wrapped present.

Why is measuring so important for AppSec? There are many reasons, but two stand out: To be effective, an AppSec program needs (1) to be customized for an enterprise’s particular environment, and (2) to expand to cover the entire application landscape. Neither of those things will happen without solid metrics.

Customize your AppSec program with metrics

An AppSec program can only truly be effective if it’s customized to address the types of vulnerabilities that emerge in your environment, and where and how they are ending up there. If you are neglecting key ways that vulnerabilities are getting into your apps, or sets of vulnerable applications lurking in your environment, then your AppSec program is not working. The only way to effectively customize a program is to understand your environment and security gaps. And you get this type of information through metrics. For instance, you need data – often derived from analytics dashboards from service providers -- that will help answer the following questions:

  • Where are the vulnerabilities? It’s important to track whether vulnerabilities are occurring with in-house code, third-party code or open source components.
  • How are vulnerabilities being found? And when? This information can help determine whether some tools or practices are more effective or more efficient than others at discovering vulnerabilities early or later in the application lifecycle.
  • Are some teams able to fix problems faster than others? Are some teams not keeping up? If not, why not?
  • What types of vulnerabilities are being seen most often? Are some more common? Is one type more common in certain systems?

AppSec is not one-size-fits-all; there are a variety of factors that will influence your particular threat landscape – including developer experience, programming language used and testing methods employed. With feedback about where and what vulnerabilities are emerging in your environment, you can tweak your system to address them more effectively. Is one team’s code showing more vulnerabilities than others? Focus developer training on that team. Is one type of vulnerability more common? Use that data to prioritize your efforts on the most prevalent flaws. Are vulnerabilities emerging in third-party code? Expand your program to include third-party apps. In the end, the more you refine your program to work with the particulars of your environment, the more improvement you will see, and the more secure you will become.

Expand your AppSec program with metrics

To effectively reduce risk, an AppSec program needs to cover all applications at an enterprise – including those built, assembled or purchased. And you won’t reach this point without the executive team’s support, and funding. An essential part of getting that support is through metrics. But not just any metrics – the key to getting executive buy-in with metrics is tweaking the numbers to speak to this audience.

To make AppSec relevant to CxOs and business units, you need to make it a part of larger initiatives, programs and issues that are important to the business. Understand what is most important to this audience: does the organization need to move faster and innovate, or save costs and increase efficiency? What customers or programs are critical to success? Use the data from your AppSec program to answer questions like the following:

  • How has the application security program improved security risk posture overall?
  • How are you helping to improve the availability and operational risk of critical systems and services?
  • How are you helping to reduce time-to-delivery for important programs or cost of operations for existing services?
  • What serious risks remain to critical operations and key products or programs?
  • How do these risks translate to threats to key customers and company reputation?
  • What are you doing to systemically reduce these risks?

When your data can answer questions like these, you prove the effectiveness of your AppSec program to C-level execs and increase your chance of getting the buy-in and support you need to scale your program and reduce risk across your entire application landscape.

In the end, solid metrics ensure you are developing the most effective program for your environment, and are able to scale your program and truly reduce risk. Find out more in SANS’ new guide, Using Metrics to Manage Your Application Security Program.

Suzanne is part of the content team at Veracode, working to create resources that shed light on AppSec problems and solutions. 

Love to learn about Application Security?

Get all the latest news, tips and articles delivered right to your inbox.