So you’ve got upper management buy-in for your application security proof of concept and are ready to start scanning applications: how do you make sure your proof of concept (PoC) is a success and that you demonstrate the need to progress to a full scale program. This article describes some of the lessons learned at the start of our large-scale deployment of Veracode within our organisation.
The first step is to socialise the PoC internally through word of mouth, discussion forums, and developer communities by driving interest in the availability of a new tool for developers, which will assist in the development process and produce better code. Ensure that you are familiar with the platform and the various IDE plugins and can demonstrate their effectiveness on a real world application (we used the OWASP WebGoat application as our technology demonstrator). The emphasis should be pro-active use of the tool to detect flaws at the point of introduction rather than as a security measurement tool. Key success factors for development teams will be the integration of the tool within common IDEs and the ease of adoption (specifically no need for in-depth product knowledge or the use of vendor specialists). Once you are familiar with the platform and your toolbox you will need some applications to scan.
The selection of which applications to scan is a key success factor: it is important that the applications chosen should be of strategic significance to the organisation in order to demonstrate the significance of the findings to senior stakeholders. Much of our difficulty arose in determining suitable applications since this information may not be readily presented in an application repository (and an application repository may be inaccurate or not exist at all). The forging of informal networks and word of mouth will be essential to success – talk to people in the canteen, be on the lookout for internal events of interest to developers; in our case we had access to an internal social media site which was an excellent platform for creating interest and awareness. Do resist the temptation to scan applications that are of low importance simply because they happen to be available, this will reduce the impact of your PoC.
Now for the moment of truth: building the code and performing the scans. It is vital that you or your team should have a working knowledge of mechanics of building software – getting access to source code is one thing, but expecting a busy development team to help you perform the requisite debug builds for scanning is unlikely to be met favourably. The ability to speak the developer’s language is a key objective in establishing an application security program and demonstrating our competence with their environments and toolchains gave us credibility when conducting the initial reviews of the scan results. Be sure to review the findings internally before distributing to the teams, and ensure that your team is familiar with the nature of the findings and can speak confidently to the risk presented by such flaws. Establishing and maintaining the credibility of your team is vital at this stage. By this stage there is certain to be a high level of interest in the PoC from within various parts of your organisation and it is important to demonstrate results as soon as possible, and indeed many teams will be eager to see their application’s results. Be sure to manage expectations around expected scan times from Veracode to avoid any possible negative perceptions of the use of a SaaS product; the emphasis should be on the ease of adoption and lack of specialised knowledge.
You should be prepared for a fair deal of scepticism around your initiative, which may be based on out-dated views of the capability of static code analysis tools, through to a belief that no problems exist within the organisation. A common objection to the use of static code analysis is the high false positive count or lack of actionable output; ensure that all applications scanned in the PoC have a readout call to demonstrate the accuracy of the findings and the specificity of the results in the flaw viewer. Negative feedback from a development team to their management could be disastrous for your future program. At the conclusion of the PoC you will need to demonstrate the value of your PoC to senior stakeholders: the key message is the rate of flaw detection that has been achieved, emphasising that such rates could not have been achieved by any manual review process. In our case we demonstrated a comparison between a Veracode scan and the traditional approach of an Application Penetration Test; the benefits in terms of cost (in our case by an order of magnitude) and timescales clearly favouring the Veracode analysis. Identifying the so called “smoking gun” is useful in demonstrating the need for a larger scale program, however be sensitive to the application team concerned and emphasise the role of your team in an advisory capacity in reducing vulnerabilities. By now you will have clearer view of the application estate within your organisation and will have an appreciation for the challenges you will face in scaling this process to a fully-fledged programme.