HP released a new tool called Scrawlr yesterday that can be used to identify certain types of SQL Injection vulnerabilities in a website. It was a joint effort with Microsoft and a direct response to the mass SQL Injection attacks of late.

Scrawlr quickly came under fire on the Web Security mailing list for having some pretty major limitations. Billy Hoffman et al have been quick to point out that the tool was designed to address a very specific subset of SQL Injection vulnerability -- the type affected by the mass attacks -- and is not designed to be a general purpose replacement for existing SQL Injection scanners. Let's look at the limitations, as outlined on the HP page, one by one.

Limitation: Will only crawl up to 1500 pages

Depends on what they mean by 1500 pages. For example, if I have these links on my front page, is that one URL or three?


Or, does it mean that it will really only crawl 1500 pages total, so if I have the same link 1500 times on the front page, it won't go any further? Either way, for most smaller websites this is probably fine. If you need more than 1500 you could give it different starting URLs in an attempt to improve coverage. It would be nice to have a clearer definition of what it means to "crawl up to 1500 pages" though.

Limitation: Does not support sites requiring authentication

Well, this will render it useless for the majority of enterprise apps. But there are still a lot of sites out there that don't require authentication, including some of the ones that got hit during the mass attacks, such as the United Nations, UK government, etc.

[Update 06/26: Thomas Ptacek Mike Tracy investigates further and provides a workaround that'll work for the majority of sites that use cookie-based auth]

Limitation: Does not perform Blind SQL injection

They have taken a lot of flack for this but Billy describes it as a conscious choice:

An early version of the tool checked for blind SQL injection, but the final verison of Scrawlr did not. ... The biggest feedback we got from early testing was developers wanted to "see" the vulnerability. Differential analysis is kind of difficult to visualize in a way that is helpful for the average dev, and pulling the table names through blind was too much of a performance issue.

I can sort of understand this rationale. Blind SQL Injection testing is much more susceptible to false positives. As users of any commercial web scanner or source code analyzer will attest, the more time you spend chasing down FPs, the less likely you are to put any faith in future results. It'd be nice if there was a way to toggle Blind SQL Injection testing on and off, though (could be off by default so nobody gets confused).

Limitation: Cannot retrieve database contents

Who cares? Find and fix the vulnerability. Pulling down the entire database "because you can" is a total ego move.

Limitation: Does not support JavaScript or flash parsing

Nobody does this very well anyway, particularly the JavaScript part. Writing a great crawler is probably the hardest part of writing an automated web scanner and it's one of the biggest differentiators from one product to the next. You're not going to get that for free.

Limitation: Will not test forms for SQL Injection (POST Parameters)

This is probably the toughest one to swallow. It's not that difficult to parse out forms from HTML, and form POSTs can represent a major chunk of the attack surface. Granted, the Chinese tool associated with the mass attacks did operate solely on GET requests (i.e. parameters in the query string) so HP can defend this again by saying the tool is really aimed at the sites being targeted by the mass attacks. I think it's a little short-sighted though; chances are that the mass attacks will evolve and it's better to be proactive about it than reactive.


It's tough to bash someone for releasing a free tool. I personally think HP should add an option for enabling Blind SQL Injection testing, and that they should consider supporting POSTs as well as GETs. You're basically getting a (massively) stripped-down WebInspect for free, so take it for what it is. No single tool is a panacea.

The jury is still out on how effective Scrawlr is against the things it does claim support for. Keep watching the Web Security list; the reviews are filtering in.

Veracode Security Solutions
Security Alternatives
Security Threat Guides

About Chris Eng

Chris Eng, vice president of research, is responsible for integrating security expertise into Veracode’s technology. In addition to helping define and prioritize the security feature set of the Veracode service, he consults frequently with customers to discuss and advance their application security initiatives. With over 15 years of experience in application security, Chris brings a wealth of practical expertise to Veracode.

Comments (8)

Andrew Becherer | June 25, 2008 11:56 am

Regarding "Limitation: Does not support sites requiring authentication" the Scrawlr supports the use of a proxy. If your site requires NTLM for authentication you could utilize the NTLM Authorization Proxy Server (ntlmaps). I have not tried this yet, as I don't have any NTLM requiring sites at the ready. It should work though.

Marcin | June 25, 2008 1:24 pm

<em>Limitation: Cannot retrieve database contents</em>

Not to mention dangerous...

Mike Tracy | June 26, 2008 2:35 pm

Why does Tom always get the credit?

CEng | June 26, 2008 2:41 pm


Doh! Sorry about that. Fixed. Maybe I should start reading the bylines. :)

Aldo Lagana | July 18, 2008 6:59 am

I have yet to play with Scrawlr, but it looks like Paros' Scanning capability.

Great blog!

dave | July 21, 2008 1:06 pm

can you recommend a free or some type of affordable tool?

DaveS | September 17, 2008 7:44 am

So what are the choices?

Acunetix WVS is WAYYY to expensive for the average Joe
WebInspect is also in the above category...

There must be something out there that a "person" (not a big business) can afford, and performs well...

Does anyone know of anything?


SQL Tutorials | April 30, 2009 9:56 pm

Does anyone know if there is another language or set of commands beside SQL for talking with databases?

I'm working on a project and am doing some research thanks

Please Post Your Comments & Reviews

Your email address will not be published. Required fields are marked *

Love to learn about Application Security?

Get all the latest news, tips and articles delivered right to your inbox.