We all know it happens, but it is rarely exposed as clearly as Adam Pennenberg did in his article for Fast Company, The Black Market Code Industry. It turns out that this 0day seller was an HP employee:

According to the consultant who snared Marester, his quarry’s skills appear quite sophisticated. His wares, if they performed as advertised, could help a hacker take down machines running that particular software anywhere in the world. His real name is Steve Rigano; he’s a self-employed network consultant from Grenoble, France, who works full time at HP, where he is listed in the switchboard and maintains an hp.com email address. He told me that he saw nothing wrong with offering tools and techniques that targeted the company providing his paycheck.

A self-taught hacker, Rigano says he discovered the vulnerabilities and coded the exploits on his own time, which he says is none of HP’s business. “I have the right to sell what I want,” he says. He told me he attracted mostly Chinese and Russian buyers, but claimed he never found takers for the HP or SAP “vulns” and exploits. He said he stopped selling black-market code in January but didn’t explain why.

Most security companies I have been acquainted with frown on this type of activity, as I am sure HP has. It’s hard for them to sell security products and services when their employees are selling the very tools the company is purportedly defending against.

[Update 7/7/2008: The information in the Fast Company article is being disputed by Steve Rigano. He has notified us that he has taken legal action against Adam Pennenberg and Fast Company.]

DWR 2.0.5 addresses an XSS vulnerability that is likely to be exploitable in most 2.0.4 installations. If your web application uses DWR’s Ajax implementation, download and install this update now!

As an aside, I’ve been a fan of DWR for a while now, not only because of its ease of integration but also because it was the first Ajax framework to offer built-in CSRF protection. You could tell that Joe Walker was taking security seriously. For this particular vulnerability, I e-mailed him on a Saturday night, and within 12 hours, he had confirmed the problem, patched the code, and built a 2.0.5 release candidate. Granted, it was a tiny code change, but I’ve still never seen a response that fast. Less than a week later, the official 2.0.5 release was tested and available for download.

That’s it for now, but I’ll be referencing this example again when I get around to writing Part 2 of my Minimizing the Attack Surface post.

This post is a response to Alan Shimel’s Topic of Interest #2 for the Security Bloggers Network.

So what motivates me to attend BlackHat? The #1 reason for me is networking — meeting new people and catching up with old friends and colleagues. Despite our best intentions, we are all busy and our networks are constantly expanding, making it increasingly difficult to stay in touch with old friends in the industry. Twitter and other forms of microblogging help you chip away at the communication gaps; you get a glimpse into peoples’ lives but it’s no replacement for a real conversation.

Obviously, the briefings themselves are a major draw. Even though it’s expanded to over 10 tracks now, the quality hasn’t suffered much. This year’s experiment with allowing paid delegates to vote on speakers seems to have produced a good lineup, though I’m sure there was still a selection committee that could and probably did overrule the votes in some cases. Either way, BlackHat presentations are a decent indicator of the overarching themes that will be prevalent in information security for the upcoming year or two.

When I first started attending BlackHat, I was drawn to the talks discussing 0-day vulnerabilities, tool releases, shellcode tricks, and the like. These days, anything relating to static analysis, automation, and of course web security are most interesting to me. I also consider who’s speaking, regardless of the topic (e.g. one of these guys presents, I’m there). In general, I’ll try to gauge how much value the speaker will add to the presentation — in other words, what do I gain by attending the talk vs. flipping through the slides later? I never attend every time slot; sometimes the hallway conversation is just more interesting.

Some of my other reasons for attending, in no particular order, most of which fall under the “networking” umbrella:

• The parties (duh)
• The Pwnie Awards
• Meeting fellow security bloggers
• Recruiting speakers for SOURCE
• Finding future Veracode employees
• Trading war stories
• Picking up vendor schwag for my kids (RSA is much better for this one)
• Meeting current and former customers — and future ones, hopefully

Things I could do without:

• The cigarette smoke
• The heat
• Quark’s

I’ve stuck around for DEFCON a couple times in the past, but I don’t anymore. I fly out Friday morning or early afternoon so I get home in time to spend the weekend with the family. Personally, three days in Vegas is plenty for me.

When it gets closer to BlackHat time, I’ll post my picks from the briefings schedule.

HP released a new tool called Scrawlr yesterday that can be used to identify certain types of SQL Injection vulnerabilities in a website. It was a joint effort with Microsoft and a direct response to the mass SQL Injection attacks of late.

Scrawlr quickly came under fire on the Web Security mailing list for having some pretty major limitations. Billy Hoffman et al have been quick to point out that the tool was designed to address a very specific subset of SQL Injection vulnerability — the type affected by the mass attacks — and is not designed to be a general purpose replacement for existing SQL Injection scanners. Let’s look at the limitations, as outlined on the HP page, one by one.

Limitation: Will only crawl up to 1500 pages

Depends on what they mean by 1500 pages. For example, if I have these links on my front page, is that one URL or three?

• http://www.veracode.com/blog/?p=111&foo=1
• http://www.veracode.com/blog/?p=111&foo=2
• http://www.veracode.com/blog/?p=111&foo=3

Or, does it mean that it will really only crawl 1500 pages total, so if I have the same link 1500 times on the front page, it won’t go any further? Either way, for most smaller websites this is probably fine. If you need more than 1500 you could give it different starting URLs in an attempt to improve coverage. It would be nice to have a clearer definition of what it means to “crawl up to 1500 pages” though.

Limitation: Does not support sites requiring authentication

Well, this will render it useless for the majority of enterprise apps. But there are still a lot of sites out there that don’t require authentication, including some of the ones that got hit during the mass attacks, such as the United Nations, UK government, etc.

[Update 06/26: Thomas Ptacek Mike Tracy investigates further and provides a workaround that'll work for the majority of sites that use cookie-based auth]

Limitation: Does not perform Blind SQL injection

They have taken a lot of flack for this but Billy describes it as a conscious choice:

An early version of the tool checked for blind SQL injection, but the final verison of Scrawlr did not. … The biggest feedback we got from early testing was developers wanted to “see” the vulnerability. Differential analysis is kind of difficult to visualize in a way that is helpful for the average dev, and pulling the table names through blind was too much of a performance issue.

I can sort of understand this rationale. Blind SQL Injection testing is much more susceptible to false positives. As users of any commercial web scanner or source code analyzer will attest, the more time you spend chasing down FPs, the less likely you are to put any faith in future results. It’d be nice if there was a way to toggle Blind SQL Injection testing on and off, though (could be off by default so nobody gets confused).

Limitation: Cannot retrieve database contents

Who cares? Find and fix the vulnerability. Pulling down the entire database “because you can” is a total ego move.

Limitation: Does not support JavaScript or flash parsing

Nobody does this very well anyway, particularly the JavaScript part. Writing a great crawler is probably the hardest part of writing an automated web scanner and it’s one of the biggest differentiators from one product to the next. You’re not going to get that for free.

Limitation: Will not test forms for SQL Injection (POST Parameters)

This is probably the toughest one to swallow. It’s not that difficult to parse out forms from HTML, and form POSTs can represent a major chunk of the attack surface. Granted, the Chinese tool associated with the mass attacks did operate solely on GET requests (i.e. parameters in the query string) so HP can defend this again by saying the tool is really aimed at the sites being targeted by the mass attacks. I think it’s a little short-sighted though; chances are that the mass attacks will evolve and it’s better to be proactive about it than reactive.

Conclusion

It’s tough to bash someone for releasing a free tool. I personally think HP should add an option for enabling Blind SQL Injection testing, and that they should consider supporting POSTs as well as GETs. You’re basically getting a (massively) stripped-down WebInspect for free, so take it for what it is. No single tool is a panacea.

The jury is still out on how effective Scrawlr is against the things it does claim support for. Keep watching the Web Security list; the reviews are filtering in.

What was the first thing you learned about network security? There’s a good chance it had something to do with port scanning. After scanning a few boxes, you realized that modern operating systems have a lot of open ports by default, meaning a lot of services. Some had an obvious purpose, like telnet on tcp/23 or ftp fon tcp/21. Others left you wondering, what the heck is listening on tcp/515 or tcp/7100? And remember, you couldn’t ask Google because it didn’t exist (well, maybe it did depending on when you got into security).

Your first real lesson about locking down a host was how to reduce its attack surface. You learned how to disable services using /etc/inetd.conf. Then you learned about rc.d and how to prevent unnecessary services from being launched at startup. Next, maybe you configured the Xserver to disallow remote connections or moved on to removing setuid permissions from files. As you worked, you’d periodically re-scan the box to gauge progress, asking yourself “have I removed everything I don’t need?” The underlying motivation, of course, is that an attacker can’t hack something that isn’t there.

You learned how to extend those concepts to the network — configuring firewall rules, router ACLs, VLANs, etc. Segmenting the network. Creating a DMZ. No need to dwell on this, you get the idea.

Eventually, people realized that applications had an attack surface too. Web servers and application servers got a lot of attention, followed closely by custom web applications. “What do you mean you can execute SQL queries against my database? That’s impossible, I have a firewall!”

Some companies, the ones who could afford it anyway, started to build security into their development cycle. Doing threat modeling during the design phase made sense, because hey, it’s much cheaper to fix security holes in a whiteboard drawing than it is to rewrite your authorization module from scratch after it’s in production.

Let’s talk strictly about custom web applications now. What I’ve observed is that most development groups, even the ones who actively engage in threat modeling, do not understand their web application’s attack surface. The lead architect can whiteboard a high-level diagram of all the major components and how they interact. Individual developers can go a bit deeper, telling you which files they touch, what database permissions they need, or how various pieces of data are encrypted in storage. At the end of this exercise you have a complete picture of the processes, data flows, protocols, privilege boundaries, external entities, and so on, and you’re well on your way to understanding all of the potential attack vectors.

Or are you?

What often gets overlooked or glossed over is the impact of external libraries or packages. Nobody writes everything from scratch. A typical list of third-party libraries for a Java-based Web 2.0 application might include DWR, GWT, Axis, and Dojo, plus about 30 other libraries to do everything from logging to parsing to image manipulation. Nine out of ten times, the libraries will be installed in full, using the default configuration from page one of the README file.

Why is this relevant? Because just as those old Unix boxes exposed unnecessary services, libraries expose unnecessary code. Let’s say you installed Dojo to simplify the process of creating an HTML table with rows and columns that can be sorted on demand. Did you remember to remove all the .js files you didn’t need? Or maybe you installed Axis or DWR or anything else that has its own Servlet(s) for processing requests. Have you compared what that Servlet can do against what you need it to do?

A fictitious example may help illustrate further. Imagine you just downloaded a new library called WhizBang. You follow the installation instructions to define and map two servlets in your web.xml file, WhizServlet and BangServlet, and you configure it to integrate with your web app. After a bit of trial and error, it’s functional. Yay! This is where most developers stop.

Nobody asks, “how much of this do I actually need?” Case in point, what if your application only uses WhizServlet? BangServlet is still exposed, and you don’t even use it! Similarly, what if WhizServlet takes an “action” parameter which can be either “view”, “edit”, or “delete”, and your application only uses “view”? You’re still exposing the other actions to anybody who knows the URL syntax (pretty trivial if it’s open source). You wouldn’t expose large chunks of your own code that you weren’t using, so why should it be any different with libraries?

This post is getting kind of long so I’m going to split it up. In the next post, I’ll continue the discussion of attack surface minimization, as well as some of the tradeoffs that go along with this approach.

I was just reading Dre’s post, R.I.P. CISSP, over at the tssci security blog, in which he predicts the upcoming OWASP People Certification Project will be the next big thing. This paragraph is quoted from James McGovern’s blog (James is the project leader):

As an Enterprise Architect, I understand the importance of the ability for a security professional to articulate risk to IT and business executives, yet I am also equally passionate that security professionals should also have the capability to sit down at a keyboard and actually do something as opposed to just talking about [it].

I agree wholeheartedly with this sentiment, and I believe the project goals are noble. So I went to read the latest OPCP draft proposal to see how they planned to tackle this admittedly difficult problem. What did I find? It’s just another test, with questions in a dozen or so broad categories. Far more specialized that CISSP, with topics that are more relevant to application security, but ultimately, still just a test.

The comment I once made about security educators/trainers is relevant here. Whatever questions end up on the OPCP test, these educators could probably answer most of them correctly without even studying. They lecture day in and day out about these topics. They have heard obscure questions and are prepared to answer them. And yet, many of them do not have any practical field experience.

A client chastised me once for making a statement that penetration testing is a mixture of art and science. He wanted to believe that it was completely scientific and could be distilled down to a checklist type approach. I explained that while much of it can be done methodically, there is a certain amount of skill and intuition that only comes from practical experience. You learn to recognize that “gut feel” when something is amiss. He became rather incensed and, in effect, told me I was full of it. This customer went on to institute a rigid, mechanical internal process for web app pen testing that was highly inefficient and, ultimately, still relied mostly on a couple bright people on the team who were in tune with both the art and the science.

Certifications only test the science.

From the Burlington Free Press, a story about a local hacking competition set up as a spectator event.

Their competition, tantalizingly called a “digital combat exercise,” was supposed to give onlookers a rare opportunity to watch a computer hacking job in progress, complete with play-by-play.

It didn’t work out that way, though, thanks to — what else? — some sort of technical glitch that obstructed efforts to monitor what the competitors were doing. So for the few non-techie spectators who showed up, the business of hacking was still as opaque and mysterious at the end of the 1 1/2-hour exercise as it was in the beginning.

A technical glitch? They always happen at the worst possible time, don’t they? Read on.

The commentary was to come from Peter Stephenson, a member of the program’s faculty, who sat at his own terminal and displayed on a big screen something he called a “sniffer trace,” a multi-colored table with columns of numbers and letters — the first in what was to be a series of tableaus that held the promise of monitoring all the traffic on the network next door.

The minutes passed, and not much happened. The sniffer trace stayed the same, and from time to time, when Stephenson tried to check on what individual teams were up to, the screen went blank. Could it be that the hackers weren’t getting anywhere?

Someone decided to check on them in the old-fashioned way — paying a visit in person. The report came back that they were, in fact, getting somewhere — finding holes and vulnerabilities of various kinds.

You’d think that somebody on the faculty, or one of the grad students, or even somebody in the audience would have realized the problem. The story implies that they never did figure out what those pesky hackers were up to.

The Verizon Business data breach report is by far the most comprehensive and detailed report on data breaches I have seen. It is great to see the break down of what is the root cause of these expensive and significant computer security failures. While it is interesting to see counts of malware infected computers from Symantec and vulnerability counts from CVE, this report gets to the actual attacks that organizations need to prevent with their security programs.

Digging into the full report they say that 59% of the breaches involve hacking. Of those the breakdown is this:

• Application/Service layer -39%
• OS/Platform layer – 23%
• Exploit known vulnerability -18%
• Exploit unknown vulnerability – 5%
• Use of back door -15%

“Attacks targeting applications, software, and services were by far the most common technique, representing 39 percent of all hacking activity leading to data compromise. This follows a trend in recent years of attacks moving up the stack. Far from passé, operating system, platform, and server-level attacks accounted for a sizable portion of breaches. Eighteen percent of hacks exploited a specific known vulnerability while 5 percent exploited unknown vulnerabilities for which a patch was not available at the time of the attack. Evidence of re-entry via backdoors, which enable prolonged access to and control of compromised systems, was found in 15 percent of hacking-related breaches. The attractiveness of this to criminals desiring large quantities of information is obvious.”

The largest single type of breach is hacking and within that the largest type is application/service layer attacks. So if we multiply 59% times 39% we get 23% of those 500, or 115, data breaches are due attackers hacking applications. That is a very significant number of the whole slice of the data breach pie. It is clear that securing applications is a significant part of protecting against data breaches.

Full Report