[Today we have our first guest blog entry from Elfriede Dustin. Elfriede is a co-author of "The Art of Software Security Testing" and has written a few books on software testing, most notably, "Automated Software Testing" published by Addison-Wesley in 1999. We have heard plenty from security experts on how to fix the software development process to produce more secure software. Elfriede brings a QA practitioners viewpoint. I'd like to hear more from the testing community on this topic. - Chris Wysopal]
The Software Trustworthiness Framework (STF©)
by Elfriede Dustin
Recently I presented the topic “Automated Software Testing” at a Department of Homeland Security workshop on Software Assurance. Most practitioners at the workshop equated Software Assurance with Software Security Testing and one might wonder what automated software testing has to do with software assurance?
In order to achieve software trustworthiness in the limited amount of time that’s generally allowed to produce software, a combination effort of automated software testing along with security testing is required: The Software Trustworthiness Framework (STF©) is needed.
Josh Bloch, chief Java architect at Google, said in a recent statement “Regardless of how talented and meticulous a developer is, bugs and security vulnerabilities will be found in any body of code – open source or commercial. Given this inevitably, it’s critical that all developers take the time and measures to find and fix these errors.”
Developers however are strapped cranking out new features while trying to meet the often unreasonable deadlines. First-to-market is key, beating the competition is the goal. Given this dilemma, where software developers alone cannot be responsible for software assurance, we need to look to other resources to help us win the software trustworthiness battle. Who is better suited to help a developer conduct security testing than the software testing groups already in place?
In the traditional software development lifecycle, software trustworthiness is often an afterthought, and security verification and testing efforts are delayed until after the software has been developed. Meeting deadlines is key, at all cost, including that of trustworthiness, yet vulnerabilities are an emergent property of software that appear throughout the design and implementation cycles.
Currently, much of the Security testing that is done after the software has been implemented, such as paying an external party to perform security testing and issue a report, can be considered just a Band-Aid solution. It is tempting for security testing teams to focus purely on the mechanics of testing the security of a software application and pay little attention to the surrounding tasks required of a secure software development lifecycle, such as automated software testing. This is where the STF© comes into play.
It is not possible to “test” software trustworthiness into software and it is widely known that the earlier a defect is uncovered, the cheaper it is to fix.
A full lifecycle approach to software development is the only way to achieve software trustworthiness. Therefore, combining security testing with test automation, and a before, during, and after approach to software development is required.
The most effective software trustworthiness programs start at the beginning of a project, long before any program code has been written. An effective security process is one that is used throughout the SDLC and one that employs automated testing technologies.
The Automated Testing Lifecycle Methodology (ATLM) described in the book “Automated Software Testing ” is a structured methodology, supports the successful implementation of automating testing, has been implemented by companies throughout the world, and is recommended by various tool vendors. The ATLM approach is consistent with rapid application development efforts including engaging the user early in the development cycle. Many universities and professional software testing organizations are adopting this methodology in their software testing courses, along with commercial companies heavily invested in automated testing.
The Secure Software Development Lifecycle (SSDL) described in Chapter 3 of the book “The Art of Software Security Testing” is a structured methodology that has emerged to support the successful implementation of secure and trustworthy software. In the SSDL security issues are evaluated and addressed early in the system’s lifecycle, during business analysis, throughout the requirements phase, and during the design and development of each software build. This early involvement allows the security team to provide a quality review of the security requirements specification, attack use cases, and software design. The team also will more completely understand business needs and requirements and the risks associated with them. Finally, the team can design and architect the most appropriate system environment using secure development methods, threat-modeling efforts, and so on to generate a more secure design. Much research has been performed in the area and resulting success of using a Secure Software Development Lifecycle (SSDL). The SSDL approach to software development is also recommended by some departments in DHS, Microsoft, and other major organizations.
Amalgamating the Automated Testing Lifecycle Methodology (ATLM) together with the Secure Software Development Lifecycle (SSDL) combines automated software testing with software security testing into the Software Trustworthiness Framework.
More on the STF©:
As outlined in Figure 1, the SSDL and ATLM represent a structured approach toward implementing trustworthy software – The Software Trustworthiness Framework. This framework mirrors the benefits of modern rapid application development efforts. Such efforts engage the stakeholders early on as well as throughout analysis, design, and development of each software build, which is done in an incremental fashion.
The inner layer of Figure 1 describes the ATLM and has six primary processes:
- 1. Decision to Automate Testing
- 2. Test Tool Acquisition
- 3. Automated testing Introduction Process
- 4. Test Planning, Design, and Development
- 5. Execution and Management of Tests
- 6. Test Program Review and Assessment
The outer layer of Figure 1 describes the SSDL and has six primary processes that are intertwined with the ATLM:
- A. Security guidelines, rules, and regulations – Oversight
- B. Security requirements: attack use cases
- C. Architectural and design reviews/threat modeling
- D. Secure coding guidelines
- E. Black/gray/white box testing
- F. Determining exploitability
The combined processes and tools make up the Software Trustworthiness Framework (STF©).
Implementing the Software Trustworthiness Framework (STF©) will allow for repeatable and consistent verification of new releases and software patches. It will evaluate the trustworthiness from an end-to-end system perspective, and will verify that the integration of components yield a trustable system.
Automating the testing efforts will allow for quick measurements of testing completeness and allow for testing consistency. Additionally, implementing the SSDL as described in “The Art of Software Security Testing” as part of the STF© will support efforts to make software secure and together with the ATLM it can assure that it is robust and behaves as expected in while interacting with multiple software applications, as required.
Use of the STF© will allow software testers to gain confidence that a fault in one piece of software does not introduce unknowns in a set of integrated software components. It will assure that the software trustworthiness is verifiable, given the often sparse level of software specification accessibility.
The Software Trustworthiness Framework provides a structured, multi-stage approach supporting the detailed and interrelated activities required to introduce and utilize automated tools along with security testing best practices with an iterative / spiral development approach. These activities include: test design development, development and execution of test cases, development and management of test data and the test environment, documenting, tracking and closing trouble reports found during testing.
 “Automated Software Testing,” Elfriede Dustin, et al, Addison Wesley, 1999
Written by: Chris Wysopal