The recurring theme in her manifesto is the notion that certain software suppliers are “too big to test”. It’s fine for the little guys, but not the 800-pound gorillas. Instead, software purchasers should blindly trust companies with security teams and assurance processes to produce secure code. If only it were that simple. In fact, according to our semi-annual State of Software Security Report, there’s negligible variation in security quality across software suppliers regardless of company size.

We’re both flattered and amused that Ms. Davidson believes our company alone “created a market” for testing the software supply chain. On the contrary, the market has created itself. Take a look through the noteworthy breaches from the past 12-24 months; software vulnerabilities have been the culprit in nearly every case. CISOs are waking up to the stark realization that all software — internally or externally produced — introduces risk into their organizations. In this day and age, wise companies harbor a healthy suspicion of their software vendors. Oracle can choose to do security testing in-house, but a company that’s “running their entire business” on Oracle’s software has a right to request unbiased evidence that the testing process is working.

That being said, Oracle is hardly the poster child for security process. Within the security community, they are notorious for shipping insecure products. Their laughable “Unbreakable” marketing campaign was famously debunked by security expert David Litchfield, who uncovered several critical (and easily avoidable) vulnerabilities within a matter of weeks. They’ve also earned a reputation for glacial response times and sloppy patches. No company can be expected to build perfectly secure software, but it’s pretty obvious why external validation is needed to complement in-house process — one need look no further than ZDI for evidence. Even Ms. Davidson’s own example illustrates how an outsourced service provider “HuiMaika’i” detected multiple vulnerabilities that weren’t discovered by Oracle’s internal team.

Perhaps the most shocking admission about Oracle’s security program is their interpretation of the “need to know” principle. Ms. Davidson asserts that she doesn’t need access to bug databases. This is a classic liability avoidance move and one that we’ve witnessed in other organizations as well. Creating barriers to vulnerability information facilitates a culture in which the executive has plausible deniability of critical bugs and can simply look the other way if a ship deadline is looming or if the auditors pay a visit. CISOs should be clamoring for as much data as they can get their hands on, not eschewing it.

Finally, Ms. Davidson seemed offended that a tenured university professor would suggest licensing software developers to create a system of accountability. Ironically, only a few years ago, she sent a letter to top universities pressuring them to incorporate secure coding guidelines such as the SANS coding certification into their curriculums. She told them, “We will start making our purchasing decisions, if you will, based on that.” Apparently, it’s OK for Oracle to flex their muscle when “buying” (i.e. hiring) from universities, but it’s not OK for Oracle’s customers to hold them to similar standards? It certainly sounds like Oracle has been feeling the pressure lately.

There are third-party tests and assessments for perhaps every important purchase in business or in our personal lives. Companies hire law firms and specialists when they make acquisitions. People look to safety and quality tests from trusted sources before they buy everything from baby strollers to cars. You wouldn’t think of buying a home without a home inspection. In each case, the cost of the independent test must be commensurate with the purchase price and the risk. Look at the typical due dilligence around home purchase. It doesn’t always make sense to pay an engineering firm thousands of dollars for a structural analysis, but it does make sense to hire a home inspector for a few hundred dollars, who in a few hours can uncover termites or a leaking roof. These are problems that must be fixed, and because the testing cost is so low it would be negligent not to do it. Most of the major software vendors have participated in third-party testing either as part of their SDLC, to vet code they were acquiring or licensing, or as part of one of their customers’ procurement process.

Veracode has never claimed that binary SAST provides complete software assurance. From the beginning, we have recommended multiple testing methods to detect vulnerabilities that static automation can’t. In fact, it’s impossible to receive our top ratings without a clean bill of health from a manual penetration test. Each layer of testing, while imperfect on its own, uncovers problems that must be corrected.

Outsourcing is not a dirty word. Many companies outsource development for entire products or components of them. Companies also outsource testing and training. The multi-billion dollar IV&V market grew out of this need — it’s simply good business. The goal is shipping secure code, not making a feel-good proclamation that your team can handle a modern development challenge with no outside help. While Oracle can be proud that they have tamed a source code tool and lived to tell the tale, other companies are securing their code faster and cheaper with the help of outsourcing. Even Veracode customers haven’t fully outsourced security; many of them have in-house security expertise and are just employing a service to make their security processes more robust. They are still full participants in the process, making decisions around how/when to remediate, how much to invest, etc. Veracode acts as an application security partner, providing customers valuable intelligence gleaned from the software ecosystem. Just as Google gets smarter with every search that it does, Veracode gets smarter with every scan we do.

At least we can rest easy knowing that Oracle would never hire lobbyists to promote an agenda. That’s a relief!

Veracode Security Solutions

Security Alternatives

Security Threat Guides

Fair warning, this is a bit of a rant.

Back in my consulting days (early 2000, I’m getting old), we delighted in the fact that our web application penetration testing methodology didn’t rely on automated tools. This was completely true; we did everything manually, and we were among the best in the industry. Many so-called security consultants of the day would run a commercial web scanner and repackage the results as a high dollar “penetration test” — what a ripoff!

What we didn’t acknowledge to our customers is that those web scanners, even in their immature state, were probably capable of detecting some of the low hanging fruit that we didn’t want to spend our time looking for. Oh, we’d find a few “representative examples” of XSS and SQL injection, but then we’d get bored and move on to the more interesting and complex attack vectors. In our naivete, we figured developers would be inspired to revisit their entire input validation and/or output encoding practices, as opposed to just fixing the proof-of-concept examples we found.

Meanwhile, the commercial web scanner vendors were always downplaying the value of manual testing! “Why would you want to pay for an expensive penetration test when you can just run this less expensive tool and find the same vulnerabilities?” They’d gloss over all the technical challenges of automated web scanning and conveniently forget to mention how it was impossible for them to find authorization issues, cryptographic weaknesses, business logic flaws, and so on.

What’s my point?

Using multiple testing methodologies is crucial. Sure, there may be some overlap, but ultimately they are complementary to one another. That’s why at Veracode, we’ve never positioned automated static analysis (SAST) as a complete solution. That’s why we integrated both automated web scanning (DAST) and manual penetration testing into our service offerings less than a year after launching the company, even though SAST is our patented bread-and-butter technology. This meant we could always be completely honest about the strengths and weaknesses of each technique. I’ve had a slide titled “There Is No Silver Bullet” in my corporate slide deck since the very beginning.

Our silver bullet is better than yours

Meanwhile, it’s been amusing to watch other companies — who only had a single offering — having to espouse the tactic of downplaying any testing approach that wasn’t in their service portfolio.

• Over at Fortify, Brian Chess famously predicted that 2009 would mark the end of penetration testing.
• Over at WhiteHat, Jeremiah Grossman often downplays the value of writing secure code and testing code quality.
• Even as recently as last week, we have Errata Security (a consultancy) claiming that automated tools are useless and doomed to fail. Welcome back to 1999.

I’m only picking on these guys because they’re visible, well-respected practitioners in the application security space. Of course Brian knows source code scanning is an incomplete solution, and now that Fortify and WebInspect are part of the same parent company, I suspect he’s adjusted his message. I’m certain Jeremiah knows there’s value in writing secure code during the SDLC, which is why WhiteHat is now trying to get into the SAST market by acquiring some technology.

And I’m pretty sure Dave Maynor knows automation does provide real value. How else can a big company — spooked by all the recent breaches — quickly hunt for SQL injection vulnerabilities across 5,000 websites without the benefit of automation? How does one look for issues in the 150 third-party libraries you use, where only the binary is available? Do you hire Mark Dowd to spend a month looking at each one?

Building trust

We all know a few sales reps that jump from one company to another, changing their pitch as they go no matter how much it conflicts with things they’ve said in the past. First a service-based approach is best, but suddenly an on-premise tool is better. Source code scanning used to be pointless, but now it’s the best thing since sliced bread! It’s no surprise these guys don’t experience more success — they lack credibility. The most successful account reps I’ve seen are the ones who build trust with their customers over time by being honest about what they are selling, even when hopping from one company to the next.

Look, it’s no big secret why people talk up their own stuff and imply everything else stinks. It’s part of the sales and marketing machine and by no means is it unique to the security industry. Even so, can’t we make an effort — as practitioners — to cut back on the rhetoric a little bit and be more honest with our customers? Customers look to us as experts to help them build their security programs, and what do we do? We oversell them on an approach that has huge gaps we pretend don’t exist. If you’re really looking out for your customers, start being more honest, and stop handing out kool-aid.

Here’s another approach: Instead of outright dismissing an effective technology or methodology just because you don’t sell it, sometimes it’s worth thinking about partnering, or even building something better. That’s why at Veracode we designed our service platform around the idea of technology integration. There is no silver bullet and there never will be.

Veracode Security Solutions

Security Alternatives

Security Threat Guides

In the realm of automated web application testing, today’s technologies fall into one of two categories, Static Application Security Testing (SAST) and Dynamic Application Security Testing (DAST). SAST analyzes application binaries or source code, detecting vulnerabilities by identifying insecure code paths without actually executing the program. In contrast, DAST detects vulnerabilities by conducting attacks against a running instance of the application, simulating the behavior of a live attacker. Most enterprises have incorporated at least one SAST or DAST technology; those with mature SDLCs may even use more than one of each.

In the past year or so, industry analysts and product vendors have become enamored with so-called “hybrid analysis” technologies. Hybrid techniques aim to correlate the results of SAST and DAST to dramatically expand dynamic coverage, prioritize the combined set of results, and reduce both false positives and false negatives. This whitepaper will examine each of these claims to give consumers technical insight into whether hybrid technologies can realistically live up to the hype.

Several observations will be described in the following sections:

• Hybrid analysis may expand dynamic coverage, but the lack of application context limits its effectiveness.
• The challenge of reliably generating URL-to-source mappings, coupled with the existence of URL rewriting, undermines the accuracy and usefulness of vulnerability correlation.
• Hybrid analysis does not reduce false positive rates; rather, it lulls users into a false sense of security by suggesting that non-correlated vulnerabilities are false positives.
• Correlation should not be equated with exploitability. Vulnerabilities should be prioritized based on severity and business impact, not based on how many scanners are capable of detecting it.

Download the full whitepaper.

Veracode Security Solutions

Veracode Security Threat Guides

Easy to Fix vs. Easy to Eradicate

Let’s start with some terminology to make sure we’re all on the same page. Sometimes people will say XSS is “not easy to fix” but what they really mean is that it’s “not easy to eradicate.” Big difference, right? Not many vulnerability classes are easy to eradicate. Take buffer overflows as an example. Buffer overflows were first documented in the early 1970s and began to be exploited heavily in the 1990s. We understand exactly how and why they occur, yet they are far from extinct. Working to eradicate an entire vulnerability class is a noble endeavor, but it’s not remotely pragmatic for businesses to wait around for it to happen. We can bite off chunks through OS, API, and framework protections, but XSS or any other vulnerability class isn’t going to disappear completely any time soon. So in the meantime, let’s focus on the “easy to fix” angle because that’s the problem developers and businesses are struggling with today.

It’s my belief that most XSS vulnerabilities can be fixed easily. Granted, it’s not as trivial as wrapping a single encoding mechanism around any user-supplied input used to construct web content, but once you learn how to apply contextual encoding, it’s really not that bad, provided you grok the functionality of your own web application. An alarming chunk of reflected XSS vulnerabilities are trivial, reading the value of a GET/POST parameter and writing it directly to an HTML page. Plenty of others are only marginally more complicated, such as retrieving a user-influenced value from the database and writing it into an HTML attribute. I contend both of these examples are easy for a developer to fix; tell me if you disagree. Basic XSS vulnerabilities like these are still very prevalent.

Of course, there are edge cases. Take this freakish example, which combines browser-specific parsing behavior with the ill-advised use of tainted input in Javascript code. Exceptions will always exist, but that doesn’t change the fact that most XSS flaws are straightforward to fix. We can take a huge bite out of the problem by eliminating these basic reflected cases, just like we started attacking buffer overflows by discouraging the use of unbounded string manipulation functions. Some will claim “developers shouldn’t be responsible for writing secure code,” which is noble and idealistic but also completely impractical in this day and age. Maybe it’ll happen eventually, but in the meantime there are fires to put out. So let’s step down from those ivory towers and impose some accountability.

Ease of Fix vs. Willingness to Fix

I’ve heard the assertion that XSS vulnerabilities aren’t getting fixed because they are difficult to fix. Asking “what percentage of XSS vulnerabilities actually get fixed and deployed to production?” is a valuable metric for the business, but it doesn’t reflect the actual difficulty of fixing an XSS vulnerability. It conflates the technical complexity with other excusesreasons why website vulnerabilities are not fixed.

At Veracode, we collected data in our State of Software Security Vol. 2 report that reveals developers are capable of fixing security issues quickly. While our data isn’t granular enough to state exactly how long it took to fix a particular flaw, we do know that in cases where developers did choose to remediate flaws and rescan, they reached an “acceptable” level of security in an average of 16 days. This isn’t to say that every XSS was eliminated, but it suggests that most were (more details on our scoring methodology can be found in the appendix of the report).

WhiteHat’s Fall 2010 study shows that nearly half of XSS vulnerabilities are fixed, and that doing so takes their customers an average of 67 days. These numbers differ from ours — particularly with regard to the number of days — but I think that can be attributed to prioritization. Perhaps fixing the XSS vulnerability didn’t rise to the top of the queue until day 66. Again, that’s more an indication that the business isn’t taking XSS seriously than it is of the technical sophistication required to fix.

At Veracode, we see thousands — sometimes tens of thousands — of XSS vulnerabilities a week. Many are of the previously described trivial variety that can be fixed with a single line of code. Some of our customers upload a new build the following day; others never do. Motivation is clearly a factor. Think about the XSS vulnerabilities that hit highly visible websites such as Facebook, Twitter, MySpace, and others. Sometimes those companies push XSS fixes to production in a matter of hours! Are their developers really that much better? Of course not. The difference is how seriously the business takes it. When they believe it’s important, you can bet it gets fixed.

Manufactured Contempt

There’s a growing faction that believes security practitioners are not qualified to comment on the difficulty of security fixes (XSS or otherwise) because we’re not the ones writing the code. The ironic thing is that this position is most loudly voiced by people in the infosec community! It’s like they are trying to be the “white knights”, coddling the poor, fragile developers so their feelings aren’t hurt. Who are we to speak for them? I find the entire mindset misguided at best, disingenous and contemptuous at worst. To be fair, Dinis isn’t the only one who has expressed this view, he’s just the straw that broke the camel’s back, so to speak. You know who you are.

Look, the vast majority of security professionals aren’t developers and never have been (notable exceptions include Christien Rioux, HD Moore, Halvar Flake, etc.). Trust me, we know it. I’ve written lots of code that I’d be horrified for any real developer to see. My stuff may be secure, but I’d hate to be the guy who has to maintain, extend, or even understand it. Here’s the thing — even though I can guarantee you I’d be terrible as a developer, most XSS flaws are so simple that even a security practioner like me could fix them! Here’s another way of looking at it: developers solve problems on a daily basis that are much more complex than patching an XSS vulnerability. Implying that fixing XSS is “too hard” for them is insulting!

That being said, who says we’re not qualified to comment on a code-level vulnerability if we’re not the one writing the fix? In fact, who’s to say that the security professional isn’t more qualified to assess the difficulty in some situations? Specifically, if a developer doesn’t understand the root cause, how can he possibly estimate the effort to fix? I’ve been on readouts where developers claim initially that several hundred XSS flaws will take a day each to fix, but then once they understand how simple it is they realize they can knock them all out in a week. Communication and education go a long way. Sure, sometimes there are complicating factors involved that affect remediation time, but I can’t recall a time where a developer has told me my estimate was downright unreasonable.

Bottom line: By and large, I don’t think developers feel miffed or resentful when we try to estimate the effort to fix a vulnerability. They know that what we say isn’t the final word, it’s simply one input into a more complex equation. Yes, developers do get annoyed when it seems like the security group is creating extra work for them, but that’s a different discussion altogether.

Ceteris Paribus

One final pet peeve of mine is the rationalization that security vulnerabilities take longer to fix because you have to identify the root cause, account for side effects, test the fix, and roll it into a either a release or a patch. As opposed to other software bugs where fixes are accomplished by handwaving and magic incantations? Of course not; these steps are common to just about any software bug. In fact, I’d argue that identifying the root cause of a security vulnerability is much easier than hunting down an unpredictable crash, a race condition, or any other non-trivial bug. Come to think of it, testing the fix may be easier too, at least compared to a bug that’s intermittent or hard to reproduce. As for side effects and other QA testing, this is why we have regression suites! If you build software and you don’t have the capability to run an automated regression suite after fixing a bug, then let’s face it, you’ve got bigger problems than wringing out a few XSS vulnerabilities.

My high school economics teacher used the term “ceteris paribus” at least once per lecture. Loosely translated from Latin, it means “all other things being equal” and it’s often used in economics and philosophy to enable one to describe outcomes without having to account for other complicating factors. The ceteris paribus concept doesn’t apply perfectly to this situation, but it’s close enough for a blog post, to wit: ceteris paribus, fixing a security-related bug is no more difficult than fixing any other critical software bug. Rattling off all the steps involved in deploying a fix is just an attempt at misdirection.

Closing Thoughts

My hope in writing this post is to spur some debate around some of the reasons, excuses, and rationalizations that often accompany the surprisingly-divisive topic of XSS. I want to hear from both security practitioners and developers on where you think I’ve hit or missed the mark. We don’t censor comments here, but there is a moderation queue, so bear with us if your comment takes a few hours to show up.

Veracode Security Solutions

Veracode Security Threat Guides

Don’t Forget Origin Checks on Cross-Document Messaging

Applications that use cross-document messaging could be unsafe if origin checking is done incorrectly (or not at all) in the message receivers. It’s important that developers writing apps that rely on postMessage() carefully check to ensure that messages originate from their own sites, otherwise malicious code from other sites could spoof rogue messages. The functionality itself isn’t inherently insecure, though; developers have used various DOM/browser capabilities to emulate cross-domain messaging for some time now. The window.name attribute has been abused, as has Javascript-driven injection of HTML and URL rewriting. There’s even a cross-platform JavaScript library called easyXDM that provides a friendly interface to these hacks.

One bright spot with regard to cross-document messaging is that older apps won’t be threatened by these issues, only new apps that are intentionally written to rely on the feature.

Local Storage Isn’t as Problematic as You Think

Local storage doesn’t appear to present major security risks, despite a lot of FUD circulating on the topic. Besides cookies, there have always been numerous ways for web apps to store data client-side through the use of plugins (Java, JWS, Flash, Silverlight, Google Gears, etc.) or browser extensions — WebKit/Safari/Chrome have supported local storage before it was even part of HTML5.

Developers should also be aware that as currently implemented, the HTML5 sessionStorage attribute can be vulnerable to manipulation from foreign sites under certain circumstances. A remote site can get a handle to a window containing a site for which a browser has data in sessionStorage. Then, the remote site can navigate to arbitrary URLs in that window, while the window will still contain its
sessionStorage. Hopefully this implementation bug will be fixed by the time the standard is final.

New Tags Increase Attack Surface

HTML5 will also support new data formats and tags such as the <canvas> and <video> tags. In-browser support for video means browser developers now have to parse historically bug-ridden file formats. This increases the attack surface of HTML5 browsers but otherwise doesn’t affect the typical web app developer. The <canvas> tag is a complex set of functionality mixing Javascript and imaging-related functions, and image parsers have historically been rife with vulnerabilities.

Developers Should Be Wary of Cross-Origin Javascript Requests

Another new feature set that’s not directly part of HTML5, but has recently been introduced, is limited support for cross-origin Javascript requests. Historically, it’s been forbidden for Javascript code to request pages from any host other than the page that served the script itself; this is part of the same-origin policy. However, the W3C’s current draft for Cross-Origin Resource Sharing provides a way to circumvent the same-origin policy using a mechanism similar to the crossdomain.xml file in Flash (i.e. the server decides which domains are allowed to access its resources).

Firefox, Safari, and Chrome currently allow cross-domain requests to be sent using XMLHttpRequest. Before the entire request is allowed to proceed, the browser sends a probe request using the OPTIONS method (instead of, for example, GET or POST) first. If the server responds to this probe with an “Access-Control-Allow-Origin” header that gives the source host permission to make the request, the browser will then resend the full request with the requested HTTP method. This is consistent with the current working draft for W3C Cross-Origin Resource Sharing.

However, IE works differently. Instead of relaxing permissions on XMLHttpRequest, it uses a new object type called XDomainRequest. Also, instead of sending a probe that replaces the normal HTTP method with OPTIONS, its probe includes the original HTTP method as well as the request body (in the other browsers, the request body is omitted).

The cross-domain-request features are actually fairly troublesome, from a security point of view. Malicious code on any site can cause probe requests to be sent to any other site, in every major browser, today. Developers need to be aware of both probe types and ensure that their applications won’t be fooled by probes. Fortunately, cookies aren’t passed in any browser’s probe request. Adding to the confusion, some of the official documentation on the topic contains reference code that is blatantly insecure. For example, in an MSDN page on XDomainRequest, ASP code is provided for setting the “Access-Control-Allow-Origin” header field to “*”. This would allow any remote site to make unauthenticated requests against that page from JavaScript, which is not advisable for most applications. Developers need to be sure they understand the dangers of creating an overly permissive access control list.

Sandbox Attribute Could Make Security Easier

One thing that may help, depending on how the standard is eventually defined and implemented, is the support for a sandbox attribute on IFRAMEs. This attribute will allow a developer to chose how data should be interpreted. Unfortunately, this design, like much of HTML, has a pretty high chance of being misunderstood by developers and may easily be disabled for the sake of convenience. If done properly, it could help protect against malicious third-party ads or anywhere else that accepts untrusted content to be redisplayed.

Always Remember Input Validation

The most important thing that developers can do is to remember basic security tenets, for example, the idea that all user input should be considered untrusted. They should learn how the new HTML5 features actually work in order to understand where they’d be tempted to make erroneous assumptions.

Veracode Security Solutions

Veracode Security Threat Guides

@dinozaizovi: Finding vulnerabilities without exploiting them is like putting on a dress when you have nowhere to go.

This clever analogy spurred a discussion about the importance of proving exploitability as a prerequisite to fixing bugs. While I agree that nothing is more convincing than a working exploit, there will always be a greater volume of bugs discovered than there are vulnerability researchers to write exploits. Don’t get me wrong — as a former penetration tester, I agree that it is fun to write exploits, it just shouldn’t be a gating factor. Putting the burden of proof on the researcher to develop an exploit is not scalable, nor does it help create a development culture that improves software security over the long term.

A related topic, and one that hits closer to home for me, is how software developers deal with the results of static analysis. Static analysis is often misunderstood, particularly by people who have only dealt with dynamic analysis (fuzzing, web scanning, etc.) or penetration testing in the past. Because static analysis detects flaws without actually executing the target application, there’s an increased likelihood of finding “noise” (insignificant flaws) or false positives. On the other hand, static analysis provides broader coverage, often detecting flaws in complex code paths that a web scan or human tester would be unlikely to find. So there’s your trade-off.

Here’s a conversation I have all too frequently, paraphrased:

Now imagine that conversation stretching out to 30 minutes or more. They could’ve fixed a half-dozen flaws already. And it’s not limited to SQL injection. For example, consider cross-site scripting (XSS):

Buffer overflows:

By now you get the picture. Many developers want proof, to the extent that they’ll sacrifice efficiency to get it. If we are to improve software over the long haul, developers must learn to recognize situations where it takes less time to patch a bug than to argue about its exploitability. On a more positive note, from someone who talks to static analysis customers on a daily basis, the tide is starting to turn in the right direction. But it is still an uphill battle.

Veracode Security Solutions

Veracode Security Threat Guides

We recently learned our database was illegally accessed and certain contact and account data were taken, including Monster user IDs and passwords, email addresses, names, phone numbers, and some basic demographic data. The information accessed does not include resumes.

Considering the well-known tendency to use the same password on multiple websites, compounded with the fact that Monster pledged a comprehensive security review after the first breach, it’s just embarrassing that they are still storing passwords in the clear.

So let’s talk about how to properly store passwords for a web application.

Use a one-way cryptographic hash

Don’t store your passwords in the clear! If you do, an attacker just needs to find one SQL Injection vulnerability and he’s got the password for every one of your users. The idea behind using a one-way algorithm is that the hash value can’t be reversed to “decrypt” the password. So how does authentication work? When a user attempts to login, you apply the same one-way algorithm to convert the user-provided password into the hash value, and then compare the two hashes. If they match, then the user-provided password was correct. At no time is the password ever stored in the clear.

Often, developers will hear the advice “use a hash” and interpret that as “run the plaintext password through MD5 or SHA-1 and store the result.” But that only solves part of the problem — the part about using an irreversible algorithm. It doesn’t protect against pre-computation. Let’s say you’ve used SHA-1 to hash your passwords, and your USERS table looks like this in the database:

USER          PASSWORD_HASH
admin         5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8
bob           fbb73ec5afd91d5b503ca11756e33d21a9045d9d
jim           7c6a61c68ef8b9b6b061b28c348bc1ed7921cb53

So if you wanted to obtain the original passwords you’d have to run a dictionary or brute force attack, hashing all possible password options with SHA-1 and comparing the output to the stored hashes. This would take a long time but eventually you’d figure some of them out. But what if you already had a list of all 8-character permutations and their corresponding SHA-1 hashes? Now all you have to do is look up the hashes, rather than computing them on-the-fly. This is the idea behind rainbow tables.

An attacker with a SHA-1 rainbow table covering 8-character alphanumeric combinations would quickly look up those three hashes and obtain the original passwords of “password”, “p4ssword”, and “passw0rd” respectively.

Use a salt

The best defense against pre-computation of raw hashes is salting. To salt a password, you append or prepend a random string of bits to the plaintext password and hash the result. You then store the salt value alongside the hash so that it can be used by the authentication routine. Look in the /etc/shadow file of any modern Unix system and you’ll see something like this:

user1:$1$lKorlp4C$RD5TSM6PaZ6oaWRVUuXT40:13740:0:99999:7:::
user2:$1$qOmA0CUm$I6IdbZDTDl6B6m7s77VPe1:13650:0:99999:7:::
user3:$1$nIEInNo5$PSxcLtvGIJArL8r2AQl74.:13749:0:99999:7:::

Let’s look at the “user1″ entry in the example above, paying attention to the second field which contains a bunch of alphanumeric characters separated by dollar signs. The first token, 1, is a version number, The second token, lKorlp4C, is the salt. The third token, RD5TSM6PaZ6oaWRVUuXT40, is the one-way hash that was calculated using lKorlp4C as the salt.

When the user attempts to login, the system passes the user-provided password along with the stored salt into the hash routine (in this case, md5crypt), and compares the result to the stored hash.

Each bit of salt used doubles the amount of storage and computation required for a pre-computed table. For instance, if we used one bit of salt — either 0 or 1 — the rainbow table would have to account for two variations of every password. Eight bits of salt require 2^8, or 256 variations of every password. Use a sufficiently large salt and pre-computation becomes infeasible. For example, the md5crypt utility uses 48 bits of salt (and for an extra layer of protection, it runs 1000 iterations of MD5 to slow down dictionary attacks).

There are a couple of common mistakes that people make with regard to salting. First, don’t use the same salt every time. If you do, you’re not really increasing the search space because the attacker only has to account for a single salt value. Second, don’t worry about protecting the salt values, they’re not secrets. The added security is derived not from the secrecy of the salt but rather by the amount it increases the resources required for pre-computation.

If you have OpenSSL installed you can play around with various salt mechanisms and see what the output looks like:

$ openssl passwd -h
Usage: passwd [options] [passwords]
where options are
-crypt             standard Unix password algorithm (default)
-1                 MD5-based password algorithm
-apr1              MD5-based password algorithm, Apache variant
-salt string       use provided salt
-in file           read passwords from file
-stdin             read passwords from stdin
-noverify          never verify when reading password from terminal
-quiet             no warnings
-table             format output as table
-reverse           switch table columns

$ openssl passwd -1 password
$1$LH1SwzJI$0ho4XuPVfGlbWIcNuGIap/
$ openssl passwd -1 password
$1$eAUtQOBh$GlvJwVsyb8In5KKkvnR0E0
$ openssl passwd -1 password
$1$PgaSiWTy$ElLh6uy83Y6T4Y70AGmV20

A quick Google search shows that there is a lot of confusion about salting.

But wait, now my password recovery feature won’t work

What’s that? You say your application has one of those “Forgot My Password” features where a user can type in their username and their current password will be sent to the e-mail address on file? Clearly, that requirement depends on passwords being stored either in the clear or using a reversible mechanism such as symmetric encryption.

The answer here is to redesign your password recovery feature. Don’t let an unnecessary requirement force you into poor security practices. If you must e-mail a password, generate a temporary password that’s only valid for a short time period, and require the user to login immediately and select a new password. This obviates the need to retrieve the original, forgotten password.

Why not just use symmetric encryption?

Instead of storing passwords in the clear, you could encrypt them using a symmetric algorithm such as AES and have the application encrypt/decrypt as needed. While this solves the plaintext storage problem, it creates a new problem: key management. Where do you store the key? How often does it change? How many people have access to it? What do you do if/when the key is compromised? And so on. The tradeoff really isn’t worth it for something that’s more elegantly solved with salted hashes.

Layered defenses

While you’re rethinking password storage, it might be a good time to consider other common flubs such as password complexity and brute-force protections.

In conclusion

• Storing passwords in the clear puts your users at unnecessary risk if (when) your application database is compromised
• Use salted hashes instead of storing passwords in a recoverable format
• Password recovery mechanisms can be implemented without needing to obtain the original password
• As with any aspect of security architecture, use layered defenses

Have fun refactoring!

Veracode Security Solutions

Veracode Security Threat Guides

The other day, we ran into an example that ended up being much more interesting than it appeared. The flaw category was Insecure Temporary Files, and the question was “should I really care about this?” The flaw we identified was in a Java application, and the offending line was something like this:

tmpFile = java.io.File.createTempFile(deploymentName, ".war");

I know what you’re thinking. You think the rest of this post is about how createTempFile() uses java.util.Random instead of java.security.SecureRandom to generate filenames, and since Random is seeded with the system time, you can work backwards to figure out the seed and use it to predict all future temporary files. That’s not it, so keep reading!

We couldn’t remember specifically what was so bad about createTempFile(), aside from using a non-cryptographic PRNG, so we checked the Java API for clues:

Creates a new empty file in the specified directory, using the given prefix and suffix strings to generate its name. … To create the new file, the prefix and the suffix may first be adjusted to fit the limitations of the underlying platform. If the prefix is too long then it will be truncated, but its first three characters will always be preserved. If the suffix is too long then it too will be truncated, but if it begins with a period character (‘.’) then the period and the first three characters following it will always be preserved. Once these adjustments have been made the name of the new file will be generated by concatenating the prefix, five or more internally-generated characters, and the suffix.

This behavior was verified with a quick test program:

$ for i in `seq 1 10`; do java createTempFile; done
/tmp/prefix53363suffix
/tmp/prefix200suffix
/tmp/prefix53898suffix
/tmp/prefix26801suffix
/tmp/prefix13687suffix
/tmp/prefix2221suffix
/tmp/prefix28661suffix
/tmp/prefix61720suffix
/tmp/prefix23104suffix
/tmp/prefix29833suffix

OK, that looks about right. It does what it says it does. One of my colleagues quickly raised the question, what happens if the generated filename already exists? So he generated /tmp/prefix0suffix through /tmp/prefix65535suffix and ran the test program again.

$ for i in `seq 1 10`; do java createTempFile; done
/tmp/prefix65536suffix
/tmp/prefix65537suffix
/tmp/prefix65538suffix
/tmp/prefix65539suffix
/tmp/prefix65540suffix
/tmp/prefix65541suffix
/tmp/prefix65542suffix
/tmp/prefix65543suffix
/tmp/prefix65544suffix
/tmp/prefix65545suffix

Uh-oh, not good. So not only does createTempFile() use a pretty small search space, but when it exhausts that space, it degrades to being 100% predictable? Decompiling the relevant portion of JRE 1.6.0_07, we can see exactly how the filenames are constructed:

private static File generateFile(String s, String s1, File file)
    throws IOException
{
    if(counter == -1)
        counter = (new Random()).nextInt() & 0xffff;
    counter++;
    return new File(file, (new StringBuilder()).append(s).append(Integer.toString(counter)).append(s1).toString());
}

public static File createTempFile(String s, String s1, File file)
    throws IOException
{
    ...
    File file1;
    do
        file1 = generateFile(s, s2, file);
    while(!checkAndCreate(file1.getPath(), securitymanager));
    return file1;
}

What this tells us is that createTempFile() is actually worse than we thought. Notice that counter is only ever assigned a random value once. As soon as it has that first random value, it simply increments from that point forward. The reason we didn’t get sequential output on our first test run was because we ran the test program 10 times, initializing counter each time. Had we put the loop inside the program, it would have generated a sequential list (try it yourself if you don’t believe me).

As luck would have it, Sun actually just fixed this problem in their latest release, Java 6 Update 11. Amazing that it went so long without being discovered. The updated function looks like this:

private static File generateFile(String s, String s1, File file)
    throws IOException
{
    long l = LazyInitialization.random.nextLong();
    if(l == 0x8000000000000000L)
        l = 0L;
    else
        l = Math.abs(l);
    return new File(file, (new StringBuilder()).append(s).append(Long.toString(l)).append(s1).toString());
}

If you’re wondering, the same bug is present in IBM Java 6 SR2, but it’s been fixed in SR3.

Returning to the original question that led us down this rathole, we came to the conclusion that yes, these types of flaws ARE worth fixing. Predictability and security rarely go hand in hand.

Naturally, people wanted to know how it was done. A Twitter blog entry provided some vague detail:

The issue with these 33 accounts is different from the Phishing scam aimed at Twitter users this weekend. These accounts were compromised by an individual who hacked into some of the tools our support team uses to help people do things like edit the email address associated with their Twitter account when they can’t remember or get stuck.

What’s interesting about that paragraph is that the celebrity account hacks were not related to the phishing attacks, as one might assume, and they had nothing to do with an exploitable vulnerability in the Twitter app itself. Just a case of somebody getting hold of an admin account. Ho-hum.

Tonight, the “hacker” explained to Wired Magazine how he did it. I’ll try to summarize the attack, but you might have to read it several times because it’s subtle and complicated. Ready? Brace yourself… He used a dictionary attack to brute force a password.

Continue reading here after you’ve picked yourself up off the floor. Here’s the money quote:

The hacker, who goes by the handle GMZ, told Threat Level on Tuesday he gained entry to Twitter’s administrative control panel by pointing an automated password-guesser at a popular user’s account. The user turned out to be a member of Twitter’s support staff, who’d chosen the weak password “happiness.”

Now let’s consider the application security best practices that Twitter could have followed when designing their service, any of which would have foiled the attack.

• Password complexity. In case you were wondering, the only restriction on Twitter passwords is a minimum length of six characters. No mixed case, no numbers, no special characters, none of that. Although they do encourage you to “Be tricky!”
• Brute-force protections. Clearly there’s no account lockout mechanism, unless of course “happiness” was at the top of the word list. While there is no perfect solution to brute force attacks, it would appear Twitter didn’t even try.
• Segregation of administrative functionality. I won’t underestimate the amount of effort required to segregate the admin interface. That being said, the attack would’ve failed if Twitter admins had to perform privileged functions via a dedicated internal interface.

Any others? Leave them in the comments.

In all fairness, it’s hard to make security a top priority in ANY company, much less a startup with overworked non-security-aware developers using an agile methodology with tight iterations (making some educated guesses here about Twitter). Ideally you want to start prioritizing security before you become an attractive target. Twitter missed the boat on that one, but I bet they’re paying attention now.

Veracode Security Solutions

Veracode Security Threat Guides

In 2000, I wrote an advisory about NTLM relaying (CVE-2000-0834). The problem turned out to be significantly larger than I originally suggested in the advisory. The attack extended to other NTLM-based authentications on other protocols and allowed general-purpose credential theft via a man-in-the-middle attack.

The SMBRelay tool was published in 2001 by Sir Dystic of Cult Of The Dead Cow, and that really took it to the next level. The protocol completely fell apart. It kicked off a number of other analyses of the NTLM protocol that finally resulted in this patch. Eight years after it’s discovery.

At least they got around to it. Thanks!

Veracode Security Solutions

Security Threat Guides