Musings on Custer's Last Stand

Let’s not mince words: this rambling diatribe from Oracle’s CSO is aimed directly at Veracode. No need for a cutesy acronym; we're the only company with true static binary analysis technology, delivered as a service. Now that we’ve got that out of the way, let’s try to cut through the rhetoric (in just over a thousand words, to boot).

The recurring theme in her manifesto is the notion that certain software suppliers are "too big to test". It’s fine for the little guys, but not the 800-pound gorillas. Instead, software purchasers should blindly trust companies with security teams and assurance processes to produce secure code. If only it were that simple. In fact, according to our semi-annual State of Software Security Report, there's negligible variation in security quality across software suppliers regardless of company size.

We’re both flattered and amused that Ms. Davidson believes our company alone "created a market" for testing the software supply chain. On the contrary, the market has created itself. Take a look through the noteworthy breaches from the past 12-24 months; software vulnerabilities have been the culprit in nearly every case. CISOs are waking up to the stark realization that all software -- internally or externally produced -- introduces risk into their organizations. In this day and age, wise companies harbor a healthy suspicion of their software vendors. Oracle can choose to do security testing in-house, but a company that's "running their entire business" on Oracle’s software has a right to request unbiased evidence that the testing process is working.

That being said, Oracle is hardly the poster child for security process. Within the security community, they are notorious for shipping insecure products. Their laughable "Unbreakable" marketing campaign was famously debunked by security expert David Litchfield, who uncovered several critical (and easily avoidable) vulnerabilities within a matter of weeks. They’ve also earned a reputation for glacial response times and sloppy patches. No company can be expected to build perfectly secure software, but it’s pretty obvious why external validation is needed to complement in-house process -- one need look no further than ZDI for evidence. Even Ms. Davidson's own example illustrates how an outsourced service provider "HuiMaika'i" detected multiple vulnerabilities that weren’t discovered by Oracle’s internal team.

Perhaps the most shocking admission about Oracle's security program is their interpretation of the "need to know" principle. Ms. Davidson asserts that she doesn't need access to bug databases. This is a classic liability avoidance move and one that we've witnessed in other organizations as well. Creating barriers to vulnerability information facilitates a culture in which the executive has plausible deniability of critical bugs and can simply look the other way if a ship deadline is looming or if the auditors pay a visit. CISOs should be clamoring for as much data as they can get their hands on, not eschewing it.

Finally, Ms. Davidson seemed offended that a tenured university professor would suggest licensing software developers to create a system of accountability. Ironically, only a few years ago, she sent a letter to top universities pressuring them to incorporate secure coding guidelines such as the SANS coding certification into their curriculums. She told them, "We will start making our purchasing decisions, if you will, based on that." Apparently, it’s OK for Oracle to flex their muscle when "buying" (i.e. hiring) from universities, but it’s not OK for Oracle’s customers to hold them to similar standards? It certainly sounds like Oracle has been feeling the pressure lately.

There are third-party tests and assessments for perhaps every important purchase in business or in our personal lives. Companies hire law firms and specialists when they make acquisitions. People look to safety and quality tests from trusted sources before they buy everything from baby strollers to cars. You wouldn't think of buying a home without a home inspection. In each case, the cost of the independent test must be commensurate with the purchase price and the risk. Look at the typical due dilligence around home purchase. It doesn't always make sense to pay an engineering firm thousands of dollars for a structural analysis, but it does make sense to hire a home inspector for a few hundred dollars, who in a few hours can uncover termites or a leaking roof. These are problems that must be fixed, and because the testing cost is so low it would be negligent not to do it. Most of the major software vendors have participated in third-party testing either as part of their SDLC, to vet code they were acquiring or licensing, or as part of one of their customers' procurement process.

Veracode has never claimed that binary SAST provides complete software assurance. From the beginning, we have recommended multiple testing methods to detect vulnerabilities that static automation can’t. In fact, it’s impossible to receive our top ratings without a clean bill of health from a manual penetration test. Each layer of testing, while imperfect on its own, uncovers problems that must be corrected.

Outsourcing is not a dirty word. Many companies outsource development for entire products or components of them. Companies also outsource testing and training. The multi-billion dollar IV&V market grew out of this need -- it's simply good business. The goal is shipping secure code, not making a feel-good proclamation that your team can handle a modern development challenge with no outside help. While Oracle can be proud that they have tamed a source code tool and lived to tell the tale, other companies are securing their code faster and cheaper with the help of outsourcing. Even Veracode customers haven't fully outsourced security; many of them have in-house security expertise and are just employing a service to make their security processes more robust. They are still full participants in the process, making decisions around how/when to remediate, how much to invest, etc. Veracode acts as an application security partner, providing customers valuable intelligence gleaned from the software ecosystem. Just as Google gets smarter with every search that it does, Veracode gets smarter with every scan we do.

At least we can rest easy knowing that Oracle would never hire lobbyists to promote an agenda. That’s a relief!

Veracode Security Solutions
Security Alternatives
Security Threat Guides

Comments (8)

Daniel Veiner | August 31, 2011 12:03 pm

I find it funny that Oracle is actually listed twice in the list of lobby spenders: once as Oracle Corp ($2,920,000) and then as Oracle America ($643,200). The total spending puts them at $3,563,200, just barely below the first place :-)

Andre Gironda | August 31, 2011 12:57 pm

"Ms. Davidson asserts that she doesn’t need access to bug databases. This is a classic liability avoidance move" You missed her point. She wants the information that bugs exist (and perhaps their faults and failure modes), but doesn't need the full details (i.e. the lines of source code that are responsible). A few app developers and build engineers need the full details -- not the whole company. She's absolutely right about this point. Open bug databases, especially in prominent ISVs, are a huge liability -- just like changelogs and everything else are. I'm not happy about all of this in-fighting. MAD was simply stating the obvious: that regulation and SaaS around appsec is bad for the ISV industry in numerous obvious ways. You have clearly failed to counter these arguments in this blog post. I've found that without a test harness (e.g. a proxy-aware or unaware client that can automatedly work the entire execution flow), it's impossible to do any legitimate appsec work beyond threat-modeling. Static analysis (especially security-focused) is not ideal except when annotations and source-sink-db customizations are used by the application developers to improve their results. How does Veracode improve its results? How do you demonstrate knowledge of coverage? All commercial DAST/SAST/IAST tools and services do one thing wrong: They only bring awareness to the issues. Beyond that, there is little future for these products and services. Oracle is already aware. They already know. Leave them alone.

Joshbw | August 31, 2011 12:58 pm

Reading MAD's rambling message it certainly wasn't lost on me that she was basically using a failed internal process to refute an external assessment. If it had been MS saying bugger off I'd feel differently - they have invested a huge amount of resources in gaining the right to say that and vuln trends to support their investment, but Oracle, not so much. The thing is, for companies of that size (MS, Oracle, etc) those of us considering the security ramifications already have a really good idea what to expect - we really don't need them scanned. Realistically if security drove database selection 100% of the world would run MS SQL server. That said, not all of her points were entirely invalid. Government mandated scanning is not effective policy for any number of reasons for example, and the more you guys scan the more awesome a target your vulnerability database seems. APT gets thrown around terribly casually, but in the same way that RSA's client list made them an awesome target for the PRC, if you guys had a similar client list you would also seem like an awesome target. That's basically true for any SaaS provider, regardless of product type - the larger a stockpile of valuable assets the more incentive someone will have to try and get access. For our own consideration of SaaS security providers the external vulnerability store is a very serious consideration (though in terms of internal need to know, I agree with your assessment of her arguement). Honestly, in terms of the idea of using scan results as a means of comparing vendors, I do question the utility. It seems to me that the results would only really be telling if the vendors were miles appart - you could use them to see if they seem to have no vulnerability awareness, some, or good awareness, but with all sorts of caveats. For example, if a large part of the app logic was in a technology that wasn't supported by the scanner a vendor could look artificially good or bad. The product design can similarly generate a large number of type 1 or 2 errors if atypical (for example, using the platform to do XSD based validation of XHRs instead of putting the input validation in the app). If static analysis results became the norm of comparisson I question whether we would actually see more secure software, or just software programmed to game the comparrison (sort of how standardized testing hasn't improved education, it just caused teachers to teach to the tests). Honestly, my big questions to vendors isn't based on vuln counts. I want to know what their SDLC is like, to describe their security response and remediation, and to see what sort of security requirements they will contractually agree to. Now don't take that as a disagreement with the utility of your offering - I honestly think you have the most realistic static analysis product on the market and I really dig the focus on remediation and education in your results dashboard.

TK | August 31, 2011 8:12 pm

Like you can trust an executive at a company with the following going on to "self-regulate": http://www.theregister.co.uk/2011/08/31/oracle_africa/

Larry Suto | September 1, 2011 12:56 am

I think it is important to have the option of an ecosystem where you can outsource scanning and static analysis because it can provide valuable information that leads to the discovery of security problems. The problem is that software security requires predictions to be be made on state transitions in some very complex execution contexts. The simple state tracing automata that we have today is woefully inadequate just ask any one who has done manual code review for any length of time. I think Veracode provides a valuable service but since static analysis still has a long way to go it is just not enough for providing a reasonable sense of assurance on any software system of significant complexity.

Hannibal Lecter | September 3, 2011 12:46 am

"Though not unusual for CSO's in the Fortune 500 at large, Davidson's lack of formal training in technology stands out among CSO's for major technology companies; her peers include former software developer John Stewart, CSO of Cisco Systems, computer forensics expert Howard Schmidt, former CSO of Microsoft, and famed cryptographer Whitfield Diffie, CSO of Sun Microsystems."

cwysopal | September 7, 2011 9:28 am

While 3rd party testing is important for people consuming software, SDLC based testing is important for people building software. Testing after the software is built certainly does not replace security processes during the SDLC. It is merely verification that those processes took place. The majority of Veracode customers are using our service during the SDLC to build secure software. Some do 3rd party testing in addition but not all. We believe enterprise application security programs cannot stop at the code you write but must include testing of the components and libraries you use to build your internal software and must include testing of software packages you purchase. Advocating for 3rd party testing does not mean we don't believe the best place for software security testing isn't the SDLC. We also don't believe automated testing is all that should be done during the SDLC to produce secure software. We think training is incredibly important and offer dozens of eLearning courses. We also think application security experts should be part of any SDLC so we have dozens of partners that can perform threat modeling, design reviews, secure architecture, and manual security testing for our customers in concert with our automation. Automated testing is part of the application security solution. It is our focus but since day one as a company over 5 years ago we have never said automated testing was a complete application security solution. It will always be people process and technology.

Larry | September 16, 2011 11:32 am

It's odd how Guys like Andre Gironda can defend Oracle avoiding any comments about facts as: - Their laughable “Unbreakable” marketing campaign was famously debunked by security expert David Litchfield - They’ve also earned a reputation for glacial response times and sloppy patches. He is right we may leave Oracle alone, completely alone...Avoiding to buy crappy Software.

Please Post Your Comments & Reviews

Your email address will not be published. Required fields are marked *

The content of this field is kept private and will not be shown publicly.