The Ontario Lottery and Gaming Corp. is in a bit of hot water after refusing to pay a $42.9 million jackpot:

According to the statement, Kusznirewicz was playing an OLG slot machine called Buccaneer at Georgian Downs in Innisfil, Ont., on Dec. 8 when it showed he had won $42.9 million.

When the machine’s winning lights and sounds were activated, an OLG floor attendant initially told Kusznirewicz to go to the “winners circle” to claim his prize, according to the statement. But other OLG employees immediately arrived and told him that the corporation would not be paying, because there had been a “machine malfunction.”

They offered him a free dinner for four at the casino’s buffet.

In a press release, OLG described the malfunction as follows:

“The single Buccaneer-themed slot machine in question is a two cent per play machine with a base game reward of $300 and an absolute maximum payout of $9,025,” the release states.

“The $42 million figure is not a possible award given this machine’s configuration and pay table settings.”

Of course the lawsuit will probably be thrown out, or OLG will settle with the guy for a lesser amount. But from a technical perspective, it’s amusing to think about what happened to cause this scenario. You can imagine the slot machine software looking something like this:

void do_spin() {
  spin_reels();
  if (winning_combination) {
    unsigned int winnings = calculate_payout_in_cents();
    send_to_display("You've won $%u!n", winnings/100);
    add_to_balance(winnings/100);
  }
}

int calculate_payout_in_cents() {
  int rv;
  if (rv = lookup_payout_amount())
    return rv;
  else
    return -1;
}

For some reason, something caused lookup_payout_amount() to return NULL, which meant calculate_payout_in_cents() returned -1, signifying an error. Then, in addition to implicitly casting the signed result to an unsigned type, do_spin() fails to check for the error condition! It assumes success and announces the payout via the slot machine’s display. In this case, the -1, represented as 0xFFFFFFFF in two’s complement, gets interpreted as an unsigned number, 4294967295, due to the implicit cast, and the display prints “You’ve won $42949672!”

Today’s lesson: remember to check your error conditions!

A few of us were hanging out in the Veracode kitchen the other day and got to discussing the idea of programmatically injecting vulnerabilities into software. This is essentially the opposite of the problem that most security vendors, including ourselves, are trying to solve — that is, detecting vulnerabilities. Clearly there’s not much business value in making software less safe, though you could imagine such a tool being used for educational purposes or a way to mass-produce QA test cases.

It sounds easy, right? Certainly it would be easy to inject the types of classic security problems that are trivially detectable. For example:

• Replace bounded string manipulation calls with unbounded ones, e.g. strncpy() becomes strcpy(), strncat() becomes strcat(), etc.
• Replace printf style calls that use %s format string specifiers, e.g. printf("%s", buf) becomes printf(buf)
• Replace scanf() calls with everybody’s favorite function, gets() (Hi AIX developers!)
• Create type mismatches and potential integer coercion issues by replacing unsigned variables with their signed counterparts

Or, on the web application side:

• Create SQL injection by replacing prepared statements with concatenated queries — not trivial but there are a limited number of database APIs, and SQL statements follow a defined syntax, so it wouldn’t be that hard
• Inject XSS by removing all calls to known output encoding routines
• Disable input validation by removing all calls to mechanisms such as regex replacement

Of course, when you start messing with input validation, you run the risk of altering the intended operation of the program. Maybe that regex replacement you removed was security-related, but on the other hand, maybe it’s performing a transformation that’s relevant to the application logic. If you didn’t care about the program actually being able to function, you could:

• Arbitrary shorten character arrays
• Use the incorrect functions to manipulate standard and/or wide strings
• Add or remove calls to free() or delete
• Swap calls to delete and delete[]
• Remove checks for null — string contents, malloc() return values, whatever you feel like
• Create off-by-one errors in loop counters and array indexes

I could go on forever but you probably get the point. The trick would be finding the ones that didn’t make the program segfault after 30 seconds of operation. Then again, is it really important that the vulnerable version of the program behaves identically to the original under normal operating circumstances? That constraint makes it more challenging; otherwise, it’s kind of a boring exercise. You could argue that it’s important if you plan to use the modified version to test a fuzzer (or other dynamic analysis tool) but not for static analysis.

Either way, you’d eventually hit the same boundaries as a vulnerability detection tool. There would still be entire classes of vulnerabilities that could not be addressed effectively. How do you create authorization bypass issues without an understanding of what’s being protected and from whom? How do you inject CSRF without knowing which functions are meaningful and which tokens/identifiers are safe to remove? And you can forget about business logic flaws entirely. Basically, it’s hard to break something so specific without a decent understanding of how that something was designed to function in the first place.

Now that my brain is sufficiently uncluttered, I can get back to doing real work.

Finally getting around to posting our materials from the talk that Chris Wysopal and I gave at BlackHat this year entitled “Static Detection of Application Backdoors.” Here are the slide deck and the accompanying whitepaper:

Also, as a proof-of-concept, we had demonstrated using IDA Pro’s scripting framework to detect one of the backdoor examples that we discussed — suspicious cryptographic API calls. Specifically, it flags calls to known encryption, decryption, and/or key management functions where a constant value is passed to a specific argument position. This can help identify situations such as an application encrypting data with a hard-coded key. We had numerous requests to post the code, so here it is:

Veracode’s binary analysis technology uses similar (but more sophisticated) techniques. We build our own intermediate representation of the binary’s data flows, control flows, and range propagation which is not based on IDA Pro. We then scan that representation for backdoors in ways similar to the cryptoconst script. However, at BlackHat you’re not allowed to promote your own products/services, so it wasn’t appropriate for us to use it for demonstration purposes.

[Today we have our first guest blog entry from Elfriede Dustin. Elfriede is a co-author of "The Art of Software Security Testing" and has written a few books on software testing, most notably, "Automated Software Testing" published by Addison-Wesley in 1999. We have heard plenty from security experts on how to fix the software development process to produce more secure software. Elfriede brings a QA practitioners viewpoint. I'd like to hear more from the testing community on this topic. - Chris Wysopal]

The Software Trustworthiness Framework (STF©)
by Elfriede Dustin

Recently I presented the topic “Automated Software Testing” at a Department of Homeland Security workshop on Software Assurance. Most practitioners at the workshop equated Software Assurance with Software Security Testing and one might wonder what automated software testing has to do with software assurance?

In order to achieve software trustworthiness in the limited amount of time that’s generally allowed to produce software, a combination effort of automated software testing along with security testing is required: The Software Trustworthiness Framework (STF©) is needed.

Daily we are bombarded by news media alerts of new security breaches whether at the private, state, or government level, the latest example, UCLA having to alert 800,000 to data breach. [1]

Josh Bloch, chief Java architect at Google, said in a recent statement “Regardless of how talented and meticulous a developer is, bugs and security vulnerabilities will be found in any body of code – open source or commercial. Given this inevitably, it’s critical that all developers take the time and measures to find and fix these errors.”

Developers however are strapped cranking out new features while trying to meet the often unreasonable deadlines. First-to-market is key, beating the competition is the goal. Given this dilemma, where software developers alone cannot be responsible for software assurance, we need to look to other resources to help us win the software trustworthiness battle. Who is better suited to help a developer conduct security testing than the software testing groups already in place?

In the traditional software development lifecycle, software trustworthiness is often an afterthought, and security verification and testing efforts are delayed until after the software has been developed. Meeting deadlines is key, at all cost, including that of trustworthiness, yet vulnerabilities are an emergent property of software that appear throughout the design and implementation cycles.

Currently, much of the Security testing that is done after the software has been implemented, such as paying an external party to perform security testing and issue a report, can be considered just a Band-Aid solution. It is tempting for security testing teams to focus purely on the mechanics of testing the security of a software application and pay little attention to the surrounding tasks required of a secure software development lifecycle, such as automated software testing. This is where the STF© comes into play.

It is not possible to “test” software trustworthiness into software and it is widely known that the earlier a defect is uncovered, the cheaper it is to fix.

A full lifecycle approach to software development is the only way to achieve software trustworthiness. Therefore, combining security testing with test automation, and a before, during, and after approach to software development is required.

The most effective software trustworthiness programs start at the beginning of a project, long before any program code has been written. An effective security process is one that is used throughout the SDLC and one that employs automated testing technologies.

The Automated Testing Lifecycle Methodology (ATLM) described in the book “Automated Software Testing [2]” is a structured methodology, supports the successful implementation of automating testing, has been implemented by companies throughout the world, and is recommended by various tool vendors. The ATLM approach is consistent with rapid application development efforts including engaging the user early in the development cycle. Many universities and professional software testing organizations are adopting this methodology in their software testing courses, along with commercial companies heavily invested in automated testing.

The Secure Software Development Lifecycle (SSDL) described in Chapter 3 of the book “The Art of Software Security Testing” is a structured methodology that has emerged to support the successful implementation of secure and trustworthy software. In the SSDL security issues are evaluated and addressed early in the system’s lifecycle, during business analysis, throughout the requirements phase, and during the design and development of each software build. This early involvement allows the security team to provide a quality review of the security requirements specification, attack use cases, and software design. The team also will more completely understand business needs and requirements and the risks associated with them. Finally, the team can design and architect the most appropriate system environment using secure development methods, threat-modeling efforts, and so on to generate a more secure design. Much research has been performed in the area and resulting success of using a Secure Software Development Lifecycle (SSDL). The SSDL approach to software development is also recommended by some departments in DHS, Microsoft, and other major organizations.

Amalgamating the Automated Testing Lifecycle Methodology (ATLM) together with the Secure Software Development Lifecycle (SSDL) combines automated software testing with software security testing into the Software Trustworthiness Framework.

More on the STF©:

As outlined in Figure 1, the SSDL and ATLM represent a structured approach toward implementing trustworthy software - The Software Trustworthiness Framework. This framework mirrors the benefits of modern rapid application development efforts. Such efforts engage the stakeholders early on as well as throughout analysis, design, and development of each software build, which is done in an incremental fashion.

Figure 1 The Secure Software Development Lifecycle combined with the Automated Testing Lifecycle Methodology (ATLM) results in the Software Trustworthiness Framework (STF)

The inner layer of Figure 1 describes the ATLM and has six primary processes:

• 1. Decision to Automate Testing
• 2. Test Tool Acquisition
• 3. Automated testing Introduction Process
• 4. Test Planning, Design, and Development
• 5. Execution and Management of Tests
• 6. Test Program Review and Assessment

The outer layer of Figure 1 describes the SSDL and has six primary processes that are intertwined with the ATLM:

• A. Security guidelines, rules, and regulations - Oversight
• B. Security requirements: attack use cases
• C. Architectural and design reviews/threat modeling
• D. Secure coding guidelines
• E. Black/gray/white box testing
• F. Determining exploitability

The combined processes and tools make up the Software Trustworthiness Framework (STF©).

Implementing the Software Trustworthiness Framework (STF©) will allow for repeatable and consistent verification of new releases and software patches. It will evaluate the trustworthiness from an end-to-end system perspective, and will verify that the integration of components yield a trustable system.

Automating the testing efforts will allow for quick measurements of testing completeness and allow for testing consistency. Additionally, implementing the SSDL as described in “The Art of Software Security Testing” as part of the STF© will support efforts to make software secure and together with the ATLM it can assure that it is robust and behaves as expected in while interacting with multiple software applications, as required.

Use of the STF© will allow software testers to gain confidence that a fault in one piece of software does not introduce unknowns in a set of integrated software components. It will assure that the software trustworthiness is verifiable, given the often sparse level of software specification accessibility.

The Software Trustworthiness Framework provides a structured, multi-stage approach supporting the detailed and interrelated activities required to introduce and utilize automated tools along with security testing best practices with an iterative / spiral development approach. These activities include: test design development, development and execution of test cases, development and management of test data and the test environment, documenting, tracking and closing trouble reports found during testing.

---

[1] http://www.securityfocus.com/news/11429
[2] “Automated Software Testing,” Elfriede Dustin, et al, Addison Wesley, 1999