“Is your scanner production-safe?” It’s one of the first questions teams ask me when we are discussing CA Veracode’s Web Application Scanning and black box testing capabilities. For many, this translates to two potential issues:
- Denial of service (DOS) – will your testing overload my application and take it down?
- Malicious attacks – if my application is susceptible to SQL injection, will your scanner drop tables in my database?
I learned first-hand that other things go wrong when you run production scans without thinking things through ahead of time – and it took something extremely serious for me to see this: fantasy sports.
I used to run a fantasy golf league with an app that I developed and hosted. Of course, when I started at CA Veracode, I wanted to test my app. I knew our dynamic scanner is production-safe, so I (foolishly) configured the scan with my commissioner account and launched it right before going to bed. I woke up to a flood of emails and texts from the people in my league outraged at the trades that had been proposed, accepted, and processed on their behalf. I learned a few things:
- The CA Veracode scanner is production-safe, since my poor excuse for hosting infrastructure held up fine and all of the attacks were benign.
- Telling a die-hard fantasy golf fan that they traded Rory McIlroy for Martin Laird makes them angry.
- There are definitely other considerations for production safety.
Below I’ll take a look at a few other questions you need to answer before scanning your applications in production.
Who might I annoy with this scan?
Does your application have a button that triggers emails, notifications, or some other business action? The CA Veracode DynamicDS scanner will click it, and most likely click it several times. Security teams have historically (and erroneously) been known as pests – don’t install that program! Change your password! That perception is shifting. People are starting to understand the importance of what security professionals do and preach. Let’s not reverse the shift by flooding our colleagues’ inboxes with emails from a scan.
What type of database clean-up will I have to do?
I learned about the database clean-up the hard way, by spending several hours reversing fantasy golf trades. This potential issue directly relates to the credentials and associated user role provided to the scanner. Scanning a web application as an administrator that can edit, delete, and add accounts will allow the scanner to iterate through all of that functionality.
Is there a pre-production environment available for testing?
The importance of finding vulnerabilities as early as possible is not a surprise to anybody. The earlier you find vulnerabilities the cheaper and easier they are to remediate. There is no question testing an application in runtime is important. However, runtime does not have to always mean production. Targeting a pre-production environment, if possible, has the dual benefit of finding vulnerabilities earlier, while also avoiding issues with notifications, database clean-up, etc.
Dynamic scanning done right
So, what’s the recommendation? As with most things involving application security, the best approach will vary from organization to organization and even team to team within an organization. With 20/20 hindsight, I know what I should have done to avoid spending a Saturday morning putting fantasy golf teams back together. Here’s the bottom line:
- Create a testing environment to run thorough dynamic scans as an authenticated user
- Use the test environment to run authenticated scans for new releases
- Routinely run unauthenticated scans against the production site
An important consideration in my revised plan is considering how my entire app relies on user-to-user interaction, which prevents me from easily introducing a test account into production. If my app were an ecommerce site that did not have interaction between users, it would have been easier to also run authenticated scans in production.
The question isn't should we test in production? The answer is yes. If you don’t have access to a pre-production environment, or you’re just looking to have a more complete AppSec program, production scanning is integral. The key question is how can we be smart about production scanning?
Before introducing dynamic testing in production, you should review user roles, the application’s functionality, and the potential impacts to data. Below are a few more items to consider to ensure production scanning becomes a successful part of your program:
- Get buy-in from all relevant teams (security, operations, development, etc.).
- Properly handle (disable, route, etc.) internal notifications or generated emails during scan windows.
- Select the appropriate user role for the scanner.
- Scan at regular intervals so the teams expect it and it becomes part of the testing process.
- Have a policy in place for how to handle findings – which ones to fix, how long it should take, etc.
Production isn’t “the end” of the software development lifecycle, and security testing doesn’t stop in pre-production either. A