Comments on: CWE/SANS Top 25 Most Dangerous Programming Errors http://www.veracode.com/blog/2009/01/cwesans-top-25-most-dangerous-programming-errors/ Application security testing, analysis, and metrics Tue, 15 May 2012 22:16:53 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Chris Wysopal http://www.veracode.com/blog/2009/01/cwesans-top-25-most-dangerous-programming-errors/comment-page-1/#comment-2523 Chris Wysopal Mon, 02 Feb 2009 18:15:01 +0000 http://www.veracode.com/blog/?p=607#comment-2523 @Tejeddine: I agree with your approach. There is more to finding security vulnerabilities then just automated static and dynamic analysis. I mention them because these are approaches people are using today to find vulnerabilities cheaply and easily and are a great starting point yet usually not sufficient. I wrote a book, "The Art of Software Security Testing", which taught traditional QA testers and developers how to use the tools and techniques they already have at their disposal for finding non-security bugs and repurpose them for security bugs. I think test harnesses and unit testers fall into this category. While I think this is a good approach I am not holding my breath. Most organizations have not invested the time and resources to build up and train an internal security testing group or trained their QA people on security testing techniques. For them using automated tools can be a great start to tackling the problem of shipping software containing the CWE/SANS top 25 programming errors. As an organization matures they can get better at more semi-automated and manual techniques. -Chris @Tejeddine:

I agree with your approach. There is more to finding security vulnerabilities then just automated static and dynamic analysis. I mention them because these are approaches people are using today to find vulnerabilities cheaply and easily and are a great starting point yet usually not sufficient.

I wrote a book, “The Art of Software Security Testing”, which taught traditional QA testers and developers how to use the tools and techniques they already have at their disposal for finding non-security bugs and repurpose them for security bugs. I think test harnesses and unit testers fall into this category.

While I think this is a good approach I am not holding my breath. Most organizations have not invested the time and resources to build up and train an internal security testing group or trained their QA people on security testing techniques. For them using automated tools can be a great start to tackling the problem of shipping software containing the CWE/SANS top 25 programming errors. As an organization matures they can get better at more semi-automated and manual techniques.

-Chris

]]> By: Tejeddine Mouelhi http://www.veracode.com/blog/2009/01/cwesans-top-25-most-dangerous-programming-errors/comment-page-1/#comment-2504 Tejeddine Mouelhi Sat, 31 Jan 2009 10:37:14 +0000 http://www.veracode.com/blog/?p=607#comment-2504 I was looking forward to have an answer to my point. I will go with thinking that I was absolutely right about what I wrote in my previous post and there was nothing you can argue about. Good for me! I thought the point of writing this blog is to discuss interesting subjects with the security community and not only to advertize about your company. Best regards, Tejeddine Mouelhi, I was looking forward to have an answer to my point. I will go with thinking that I was absolutely right about what I wrote in my previous post and there was nothing you can argue about. Good for me!
I thought the point of writing this blog is to discuss interesting subjects with the security community and not only to advertize about your company.

Best regards,
Tejeddine Mouelhi,

]]> By: Tejeddine Mouelhi http://www.veracode.com/blog/2009/01/cwesans-top-25-most-dangerous-programming-errors/comment-page-1/#comment-2449 Tejeddine Mouelhi Mon, 19 Jan 2009 10:26:49 +0000 http://www.veracode.com/blog/?p=607#comment-2449 Hello, It is interesting to see that you consider only two ways to perform security testing: automated tool, and manual penetration testing. Why don't you consider semi-automated emerging tools like those based on JUnit, like for example JWebUnit, Selenium etc, which provide a good framework for writing security tests. Automated tools are not efficient for detecting subtle flaws (i suggest reading this interesting study about the limitation of automated tool - http://portal.acm.org/citation.cfm?doid=1029911 ). On the other hand, penetration testing can help detect flaws. But that really depends on the expertise of the tester, and when done manually it cannot be covering all parts of the code (consider large web apps), and will only reveal a small piece of existing flaws. I can see that this good enough for vendor, to say that they were able to find many flaws, but does that offer any guarantee of the absence of other flaws ? A good trade-off between the two approaches, would be to use semi-automated tool, to manually write tests but helped by good framework. I looking forward to read you opinion on this, as a security expert. Kind regards, Hello,

It is interesting to see that you consider only two ways to perform security testing: automated tool, and manual penetration testing.
Why don’t you consider semi-automated emerging tools like those based on JUnit, like for example JWebUnit, Selenium etc, which provide a good framework for writing security tests.

Automated tools are not efficient for detecting subtle flaws (i suggest reading this interesting study about the limitation of automated tool – http://portal.acm.org/citation.cfm?doid=1029911 ).

On the other hand, penetration testing can help detect flaws. But that really depends on the expertise of the tester, and when done manually it cannot be covering all parts of the code (consider large web apps), and will only reveal a small piece of existing flaws. I can see that this good enough for vendor, to say that they were able to find many flaws, but does that offer any guarantee of the absence of other flaws ?

A good trade-off between the two approaches, would be to use semi-automated tool, to manually write tests but helped by good framework.

I looking forward to read you opinion on this, as a security expert.

Kind regards,

]]>