TechTarget has another name for this process: monkey testing. Why? Because ad-hoc application testing typically happens without a plan of action or documentation and is often performed by users who aren't familiar with an app's intended purpose. In some cases this can be useful (for example, sniffing out unexpected flaws or code issues), but it points back to the humorous definition: get a monkey, stick him in front of a desktop or smartphone and then let him bang away at your new app. Don't record anything he's done, and there you have it — testing via the ad-hoc method. It's no wonder hackers love to get their hands on ad-hoc tested software, as they are able to penetrate the large majority of new apps on their first try. After all, without proper testing and analysis, applications are nothing more than bundles of code that are waiting to be cracked.
A Healthy Outlook
Consider the case of Healthcare.gov. When it launched a year ago, it was greeted with some fanfare, but mostly met scorn from users who found it difficult to access, slow to load and absolutely awful if they tried to get anything done. Speculation ran rampant that the government hadn't bothered to properly test the site, let alone prepare well enough for the massive crush of users who tried to log on. According to a recent Engadget article, the administration has now created a "Digital Service Team" made up of the same people — like former Google staffer Mikey Dickerson — who saved Healthcare.gov from the scrap heap last year. While the idea is for Dickerson's team to create digital standards for government sites and applications, in reality they'll spend most of their time fixing what's already broken and convincing citizens that it's safe (and maybe even enjoyable) to visit a .gov website.
This is where companies don't want to land: driven by past problems to create a "failure team" that fixes new problems as they arise. It's not just customer confidence but also internal IT effectiveness that gets compromised in this model — but is it possible to avoid?
Here's the Plan, Stan
Want to avoid ad-hoc testing? Develop a plan. If you're going to do it in-house, the plan needs to be consistent and detailed. For example, testing needs to take place across all user levels. If a test plan only considers high-access users, app-breaking bugs for frontline users may slip through the cracks. Industrious attackers may, in turn, exploit these cracks and infiltrate your network. Starting a test plan early also creates a regression portfolio — when you upgrade to a new version or migrate an app across devices, you'll have a solid foundation showing what worked, what didn't, and why. But perhaps the most critical aspect of a testing plan is uncovering problems that won't crop up in typical-use scenarios. It's these scenarios that hackers and security evaluations always try first, in hopes that you haven't done your homework or have made a serious mistake. If 99 percent of your app works perfectly, then testing needs to focus on the one percent left over. "Good enough" is never good enough.
Catch a Break, Jake
While all-encompassing test plans are the ideal solutions to eliminate ad-hoc vulnerabilities, the truth is that most companies can't afford the time or resources. In some cases, full testing on in-house apps simply isn't feasible in terms of time, while in others, the sheer volume of third-party apps makes it a Herculean task to round up problem programs for testing. As a result, it's often worth considering cloud-based, automated testing services which go after your internal, open-source and mobile applications with the same ferocity as a hacker. Of course, this is just the end game — the best solution will integrate with your existing development process to permit automated testing and reporting every step of the way.
Apps are the new vulnerabilities — everything is as-a-service and therefore susceptible, especially if ad-hoc testing stands in for more rigorous methods. Don't get breached by bugs in new code or hidden problems in old standbys: get tested, and then give up the ad-hoc.