So you’ve got upper management buy-in for your application security proof of concept and are ready to start scanning applications: how do you make sure your proof of concept (PoC) is a success and that you demonstrate the need to progress to a full scale program. This article describes some of the lessons learned at the start of our large-scale deployment of Veracode within our organisation.

Socialising the Proof of Concept

9745381_sThe first step is to socialise the PoC internally through word of mouth, discussion forums, and developer communities by driving interest in the availability of a new tool for developers, which will assist in the development process and produce better code. Ensure that you are familiar with the platform and the various IDE plugins and can demonstrate their effectiveness on a real world application (we used the OWASP WebGoat application as our technology demonstrator). The emphasis should be pro-active use of the tool to detect flaws at the point of introduction rather than as a security measurement tool. Key success factors for development teams will be the integration of the tool within common IDEs and the ease of adoption (specifically no need for in-depth product knowledge or the use of vendor specialists). Once you are familiar with the platform and your toolbox you will need some applications to scan.

Application Selection

The selection of which applications to scan is a key success factor: it is important that the applications chosen should be of strategic significance to the organisation in order to demonstrate the significance of the findings to senior stakeholders. Much of our difficulty arose in determining suitable applications since this information may not be readily presented in an application repository (and an application repository may be inaccurate or not exist at all). The forging of informal networks and word of mouth will be essential to success – talk to people in the canteen, be on the lookout for internal events of interest to developers; in our case we had access to an internal social media site which was an excellent platform for creating interest and awareness. Do resist the temptation to scan applications that are of low importance simply because they happen to be available, this will reduce the impact of your PoC.

Building and Scanning

Now for the moment of truth: building the code and performing the scans. It is vital that you or your team should have a working knowledge of mechanics of building software – getting access to source code is one thing, but expecting a busy development team to help you perform the requisite debug builds for scanning is unlikely to be met favourably. The ability to speak the developer’s language is a key objective in establishing an application security program and demonstrating our competence with their environments and toolchains gave us credibility when conducting the initial reviews of the scan results. Be sure to review the findings internally before distributing to the teams, and ensure that your team is familiar with the nature of the findings and can speak confidently to the risk presented by such flaws. Establishing and maintaining the credibility of your team is vital at this stage. By this stage there is certain to be a high level of interest in the PoC from within various parts of your organisation and it is important to demonstrate results as soon as possible, and indeed many teams will be eager to see their application’s results. Be sure to manage expectations around expected scan times from Veracode to avoid any possible negative perceptions of the use of a SaaS product; the emphasis should be on the ease of adoption and lack of specialised knowledge.

Common Criticisms

15431307_sYou should be prepared for a fair deal of scepticism around your initiative, which may be based on out-dated views of the capability of static code analysis tools, through to a belief that no problems exist within the organisation. A common objection to the use of static code analysis is the high false positive count or lack of actionable output; ensure that all applications scanned in the PoC have a readout call to demonstrate the accuracy of the findings and the specificity of the results in the flaw viewer. Negative feedback from a development team to their management could be disastrous for your future program. At the conclusion of the PoC you will need to demonstrate the value of your PoC to senior stakeholders: the key message is the rate of flaw detection that has been achieved, emphasising that such rates could not have been achieved by any manual review process. In our case we demonstrated a comparison between a Veracode scan and the traditional approach of an Application Penetration Test; the benefits in terms of cost (in our case by an order of magnitude) and timescales clearly favouring the Veracode analysis. Identifying the so called “smoking gun” is useful in demonstrating the need for a larger scale program, however be sensitive to the application team concerned and emphasise the role of your team in an advisory capacity in reducing vulnerabilities. By now you will have clearer view of the application estate within your organisation and will have an appreciation for the challenges you will face in scaling this process to a fully-fledged programme.

About Colin Domoney

Originally a hardware engineer working on secure communications embedded systems in South Africa, my more recent experience has been in developing high availability medical systems, and leading a large scale application security programme in a large multinational. Experienced in most modern programming languages my current focus is in facilitating remediation across our application estate.

Comments (2)

Arved | December 12, 2013 2:28 am

Great summary ! It reflects to 100 % my experience, happy to have a more in-depth chat to share ideas about what can be done to help the customer keeping the motivation high implementing a Application security initiative in development ! For the POC let me add a few thoughts on top.

- To get the buy in for possibly an Sdlc initiative with a proper risk management approach make sure that you get from the beginning the CISO , the Head of Development , the QA team and eventually the business owner into the POV. Implementing an SDLC will need the buy in from a lot of stake holders.

- It will help the customer if he picks the strongest developers for the POC, they are not afraid of being pointed out for security issues. Weak developers tend to oppose against it which leads to hiding the results or playing them down.

- Don't take too small applications - if only one or two small ones are getting scanned and suddendly the found number of issues in the POV is too small the customer might think that he is secure - simply because the found number of issues might be not relevant enough. make sure that the minimum number of LOC are 300.000 - you don't want it to be too big either so try to limit it to a few MB.

- take the time to explain that this will not be an easy attempt to get the initiative implemented and position the services and an sdlc workshop as a follow up action to help the customer in finding the right actions for his maturity level.

Great article !

ndupaul | January 29, 2014 12:30 pm

Arved, thanks so much for sharing your experiences as well, you make some great points!

Please Post Your Comments & Reviews

Your email address will not be published. Required fields are marked *

Love to learn about Application Security?

Get all the latest news, tips and articles delivered right to your inbox.