The Verizon Business data breach report is by far the most comprehensive and detailed report on data breaches I have seen. It is great to see the break down of what is the root cause of these expensive and significant computer security failures. While it is interesting to see counts of malware infected computers from Symantec and vulnerability counts from CVE, this report gets to the actual attacks that organizations need to prevent with their security programs.

Digging into the full report they say that 59% of the breaches involve hacking. Of those the breakdown is this:

• Application/Service layer -39%
• OS/Platform layer – 23%
• Exploit known vulnerability -18%
• Exploit unknown vulnerability – 5%
• Use of back door -15%

“Attacks targeting applications, software, and services were by far the most common technique, representing 39 percent of all hacking activity leading to data compromise. This follows a trend in recent years of attacks moving up the stack. Far from passé, operating system, platform, and server-level attacks accounted for a sizable portion of breaches. Eighteen percent of hacks exploited a specific known vulnerability while 5 percent exploited unknown vulnerabilities for which a patch was not available at the time of the attack. Evidence of re-entry via backdoors, which enable prolonged access to and control of compromised systems, was found in 15 percent of hacking-related breaches. The attractiveness of this to criminals desiring large quantities of information is obvious.”

The largest single type of breach is hacking and within that the largest type is application/service layer attacks. So if we multiply 59% times 39% we get 23% of those 500, or 115, data breaches are due attackers hacking applications. That is a very significant number of the whole slice of the data breach pie. It is clear that securing applications is a significant part of protecting against data breaches.

Full Report

We were more than pleased to read a new report by John Pescatore of Gartner recommending that security managers adopt the use of the Common Vulnerability Scoring System (CVSS) to support more repeatable, fast-acting vulnerability management processes.

This recommendation backs up the decision made by our CTO, Chris Wysopal, more than a year ago to adopt the CVSS standard as a part of the Veracode rating system.

Another interesting recommendation in the report is: “Enterprieses should ensure that processes are in place to detect, assess, and manage each software vulnerability class.” You’ll need a combination of static, dynamic and manual testing to do it all.

Gartner requires you to have a login to read the entire article.

On a side note, we are now linking to Technorati:
Technorati Profile

Earlier this week, I attended the first PCI Community Meeting in Toronto, a gathering organized by the PCI Security Standards Council to bring QSAs, ASVs, and other PCI stakeholders together in one room with the PCI Council. Let’s be honest here — in the security industry, discussing regulatory compliance is about as dull as it gets. On the other hand, compliance is also a major catalyst, sometimes the only catalyst, in convincing organizations to improve their security posture, so it’s important to understand. As might be expected, I focused my attention on the sessions dealing with upcoming application security requirements and standards. Of particular interest were the two panel discussions covering the Payment Application Data Security Standard (PA-DSS) and Requirement 6.6 of the PCI Data Security Standard (PCI-DSS).

PA-DSS

The PA-DSS is basically an extension of Visa’s Payment Application Best Practices (PABP) guidelines. Essentially, it’s a standard that will be applied to any application that stores, processes, or transmits cardholder data in a merchant or service provider environment. This is not to be confused with the PCI-DSS, which applies to entities/companies as opposed to applications. The PCI Council will release a draft of the PA-DSS in October for comments and suggestions, leading to a final version by the end of Q4. They are currently working on the certification process for existing QPASPs (Visa’s certification for PA assessors) to become PA-QSAs by Q2 2008.

The panel outlined the changes to PABP as follows:

• Explanation/Clarification. Just providing additional text around the best practices, presumably to make them easier to understand.
• Enhancements. Things necessary to turn a Best Practices document into a standard. The panel did not go into exhaustive detail on the enhancements, but the examples given were requiring assessors to use forensic tools to examine the output of the PA, introducing stronger lab requirements, and requiring a penetration test of the PA to identify common (e.g. OWASP-style) vulnerabilities.

Unfortunately, this was the extent of the detail. I wanted more information on the enhancements but didn’t get a chance to ask a question due to time constraints. Specifically, it would be nice to understand whether code analysis would be an alternate option for the penetration test requirement. Or what exactly they meant by using forensic tools to examine the application’s output.

Other relevant questions that came up had to do with versioning and recertification. For example, if a PA vendor releases a minor update, what is required in order to recertify the application? Would they be required to conduct another pen test, even if all they did was add a minor feature, such as the ability to recognize a new type of gift card? The panel’s response was that if a recompile was required then recertification would be necessary. However, that recertification effort could be targeted towards the new functionality. The other good question was under what lab conditions would the application be tested — the vendor or the customer. If you consider applications such as SAP, PeopleSoft, etc. which tend to be highly customized for each customer install, does it make sense to test in a vendor lab only, with no context of how the application will be deployed and configured in a production environment?

Requirement 6.6

Requirement 6.6 of the PCI-DSS becomes mandatory in June 2008 and requires all web-facing applications to either undergo a code review OR be protected by a web application firewall. There were questions on what “web-facing” means, and the Council admitted that it needed rewording. This could be interpreted as any application using HTTP as a transport mechanism, or it could mean only those applications that were Internet-facing. It sounded as if they meant the latter, but they did not say so outright which was confusing. Web services applications would also be within scope. Many people were confused about the requirements for a WAF, specifically because there are no guidelines on what functionality the WAF must have, how it must be configured, etc. Joe Lindstrom, a panelist who runs Symantec’s compliance practice, outlined these features to look for when selecting a WAF:

• Layer of defense for known attacks (e.g. OWASP Top 10)
• LDAP and/or Active Directory integration
• Logging and monitoring capabilities
• Minimal performance impact

From a security perspective, I am not sure why performance impact was on the list. As long as the security requirements are met, it should be up to you to decide how much of a performance hit your application can handle. Another panelist, Dave Wichers from Aspect Security, suggested that a WAF should also do egress filtering to detect sensitive information such as track data leaving the network.

WAFs can be configured in so many different ways, however, that people were really looking for detailed guidelines on what would satisfy the requirement. First, WAFs can be run in “detect” or “prevent” mode. The former simply logs anomalies, while the latter actively blocks traffic once it detects those anomalies. Obviously, this is a big difference from an operational risk perspective, and most organizations use “detect” mode for that reason. Secondly, the effectiveness of a WAF is limited if it is not tuned specifically for the web application that it is protecting. In order for it to know what constitutes malicious traffic, it needs to understand what normal application traffic looks like. It’s kind of like tuning an IDS, except at the application layer. Of course, this means that a certain amount of retraining is required whenever the application is updated or changed, so again, WAFs are commonly deployed without proper tuning in order to avoid unforeseen disruptions.

One panelist, Kennet Westby from Coalfire Systems, pointed out that Requirement 6.6 should not be an either-or option, since code review is an integral part of the SDLC. This time, I made it up to the microphone before time expired, and asked if the council had considered making code review the sole requirement but allowing a WAF to be used as a compensating control. In my view, this seemed more closely aligned with the intent of PCI as a security standard. Incorporating code review into the development process is the best practice, whereas a WAF is essentially just a band-aid, not a replacement for code review. Granted, a SDLC cannot be put in place overnight, and a WAF can be effective as quick fix — something is better than nothing — but it certainly shouldn’t be an equivalent way to satisfy the requirement. Dave Wichers responded to my question, agreeing that code review was the “right way,” but added that a code review is not a viable option for many companies, thus they had to offer WAF as an alternative. Time ran out, so I’m still not clear exactly what he meant by that, and I need to follow up to get more clarity on his comment. Still, I saw some heads nodding and received some positive feedback from a couple people in the audience, so I guess I’m not the only one thinking along these lines.

The panel also brought up the question of whether automated code analysis solutions could be used to satisfy the code review requirement. The general consensus on stage was that you still needed a security professional in the loop due to the false positive (FP) and false negative (FN) problem. This isn’t to say that companies shouldn’t use these tools to help them achieve compliance, but that the tools alone would not satisfy the requirement. While the panel did not recommend any specific products, they suggested that organizations should focus on three main factors when selecting code analysis vendors:

• Maturity of the product
• Ease of integration into the SDLC
• FP and FN rates as an accuracy benchmark

The idea of benchmarking for FP/FN rates shows a level of maturity that this industry is undergoing. It would be an eye-opening exercise to apply these standards to human code reviewers and human penetration testers as well. In fact, I think people would be shocked when they realize the FN levels they are accepting today.

Staying on the topic of benchmarking for a moment, the WAF discussion is interesting. What percentage of attacks do people think these are actually stopping? Is it 20% or 80%? I haven’t ever seen any solid data here, though anecdotally, the reviews certainly have not been glowing. This is a big of an unknown as FP/FN rates of code analysis tools. If automated code analysis is going to be held up to benchmark scrutiny, then WAFs and manual assessments need to be as well.

Conclusions

Overall, I found that the sessions were probably a little bit on the short side. At each session I attended, there was never enough time at the end to address audience questions. Many of the audience members were from companies working toward PCI compliance, so the questions tended to revolve around interpretation of the PCI language and intent (Is such-and-such considered a compensating control? What do you mean by connected entities?), leaving less time for discussion of the technical merits of the security requirements.

The Web Application Security Consortium (WASC) just published statistics on the prevalence of various web application vulnerabilities. The list was compiled from 31,373 automated assessments performed during 2006 by four contributing companies, with the methodology around data collection described as follows:

The scans include a combination of raw scan results and results that have been manually validated to remove false positive results. The statistics do not include the results of any purely manual security audits (aka human assessments).

As with any statistical data, the results of this study should be digested with a healthy dose of skepticism and a solid understanding of the sampling bias. Take, for example, a political tracking poll conducted by phone during normal business hours. The results of the poll will only account for the opinions of voters with publicly listed phone numbers who happen to be home during the day (and who don’t screen their calls to weed out tracking polls). The sampling bias of the WASC study is that it only accounts for the findings of automated web application scanners. As a result, it primarily reflects the capabilities and limitations of these scanners, not the general state of web application security, as one might reasonably expect from a WASC publication.

Keeping this bias in mind, what does this data really tell us, beyond the fact that automated vulnerability scanners find a lot of XSS? Does it give us true visibility into the actual prevalence and distribution of vulnerabilities in custom web applications? My answer is no.

Let’s look at a sample of the prevalence data:

• 100,059 instances of XSS across 26,531 web apps
• 23 instances of Insufficient Authorization across 4 web apps
• 84 instances of Insufficient Authentication, all occurring in 1 web app
• 426 instances of Path Traversal across 374 web apps

Those numbers just don’t pass the “giggle test.” The category that stands out the most in that list is Insufficient Authorization, a very common vulnerability in my experience. It’s highly unlikely that only four of the applications contain authorization-related vulnerabilities. All this does is highlight the limitations of automated web app scanners.

What about Cross-Site Request Forgery? That doesn’t show up at all on the list, despite the fact that the vast majority of web applications are vulnerable to it (even Jeremiah agrees on this point). It’s not on the list because it isn’t something the automated scanners can detect with any degree of accuracy. For the same reason, several categories on the OWASP Top Ten aren’t even represented, such as Buffer Overflows and Denial of Service.

Now let’s talk about false positives. The methodology clearly states that the data is a mixture of raw scan output and manually validated results. Since the results are presented in aggregate, it is impossible to derive real meaning from the figures without insight into the following information:

• Which results came from which product
• Which results have been manually validated
• The historical false positive rates, by category, for each product

There is also lack of clarity around the definition of “one vulnerability.” Consider this code snippet:

Map params = request.getParameterMap();
PrintWriter pw = response.getWriter();
for (String key : params.keySet())
  for (String value : params.get(key))
    pw.println(key + "=" + value + " ");

An automated scanner might report that as 100 different XSS vulnerabilities, one for each parameter that it fuzzed. However, there is only one actual flaw in the code. This is a simplistic example, but I suspect the inflated XSS numbers are partly due to this type of accounting.

In conclusion, here are the key takeaways from this list, after accounting for all of the weaknesses inherent to the methodology and the data itself:

• Automated web app scanners find a lot of XSS and SQL Injection
• Automated web app scanners are ineffective at finding vulnerabilities that require some understanding of higher-level logic, e.g. Insufficient Authorization or CSRF
• Including raw scan results from a category of products that are notorious for high false positive rates makes the resulting statistics even less meaningful
• The many-to-one mapping of vulnerabilities to actual instances of flawed code artificially inflate the prevalence of certain categories

In other words, this study provides minimal value to a veteran pen tester, and is misleading to just about anyone else.

You see, Oliver…
[sung] In this life, one thing counts
In the bank, large amounts
I’m afraid these don’t grow on trees,
You’ve got to pick-a-pocket or two.
You’ve Got To Pick-a-Pocket or Two lyrics, from Oliver!

Does this ABC News story on criminals looting 401K and online trading accounts of tens of millions of dollars surprise anyone in the security field? Well of course it shouldn’t. We have known about the potential for this type of criminal activity for over 8 years.

We are performing computing that requires high assurance, such as managing an online trading account, on a low assurance terminal, the home computer. The network layer of these transactions is using high security crypto in the form of SSL. The financial institution very likely has excellent security on the servers that run their account management software. But what about the customer? Who is making sure his or her machine is up to the task of being part of a high assurance transaction? The answer is nobody. Attackers will always go for the weak link and that weak link is, by far, the end user’s computer.

I gave a presentation at the Digital Commerce Society of Boston on April 6th, 1999. It was titled, “Client Security: You’ve got armored trucks, but what about the pick pockets?” Here is the presentation abstract from the announcement:

Everyone in ecommerce these days is peddling better vaults for stores and stronger armored cars to deliver payments and merchandise. Does this really matter in an Internet world where you can pick the pocket of a consumer? Or more likely, to automate the pocket picking of a large number of consumers.

Current authentication and purchasing systems rely on consumers using off the shelf operating systems such as Windows 95/98. This is the operating system which Microsoft has admitted to having no security model. Current ecommerce client security is layering strong encryption on this bed of jello.

What are some of the attacks that are being used? What technology can be used to overcome this problem?

In June 1999 I was interviewed on Dateline NBC by Lea Thompson. The story was about how banks were moving into the world of online banking without considering the risks of the customer’s home computer. It’s as if the banks were only setting up their branches in high crime neighborhoods, knowing there were dozens of skilled pick pockets waiting outside the door, and doing nothing to protect customers.

Catherine Allen, the CEO of BITS, a consortium of 100 of the largest financial institutions, was on the program defending financial organizations and stating that online banking was secure.

The final participants in the show were 2 young men in a dark room in San Francisco that agreed to demonstrate how a Trojan keystroke logger worked. The correspondent, Lea Thompson, sat at her home computer. The only information the young men knew was her email address. They sent her an email with a Trojan animated greeting card attached. It contained a keystroke logger. She opened the attachment. She then proceeded to log into her online checking account. Then she logged out. The young men in San Francisco retrieved her password from the keystroke log and proceeded to log in and check her balance. They decided not to transfer any money because it might be considered wire fraud.

It was clear to any viewer of this Dateline NBC show 8 years ago that this was a problem. So where have we gotten in the intervening 8 years? Online crime has grown dramatically and most of the attacks are taking place on the customer’s home computer –- a threat that was clearly demonstrated to banks and customers on national television.

Symantec’s latest Internet Security Threat Report has the sobering statistics showing that the “pick pocket” risk to online banking is getting worse.

• Home users were the most highly targeted sector, accounting for 93 percent of all targeted attacks.
• Eighty-six percent of the credit and debit cards advertised for sale on underground economy servers known to Symantec were issued by banks in the United States.
• The volume of Trojans in the top 50 malicious code samples reported to Symantec increased from 23 percent to 45 percent.
• Trojans accounted for 60 percent of the top 50 malicious code samples when measured by potential infections.
• Keystroke logging threats made up 79 percent of confidential information threats by volume of reports, up from 57 percent in the first half of the year and 66 percent in the second half of 2005.
• Seventy-eight percent of malicious code that propagated did so over SMTP, making it the most commonly used propagation mechanism.

So it is clear the combination of user behavior, home computer security, email, and online banking lead to massive amounts of online theft.

Back in 1999 at my talk at the Digital Commerce Society of Boston I recommended that banks require authentication tokens for accounts with more than $2000. A token doesn’t completely solve all of these problems but it can at least cut down on one of the biggest which is keystroke logging of passwords. This recommendation was dismissed as too expensive at the time. I recommended building software more securely which is starting to become an accepted practice but it needs to become a requirement.

What did we mainly do over the last 8 years? We upgraded operating systems and browsers. We added anti-virus, personal firewalls, and anti-phishing protection. We learned not to open certain attachments. Yet the problem got worse.

It is going to take a dramatic change in what we think of as secure authentication and a dramatic change in how we build software to turn this around. If we don’t do these things I fear in 2015 we will be looking at cybercrime rates that have either gone through the roof or people will have stopped trusting the Internet to manage financial accounts.