Why Test Realism Matters Or When traffic goes up and websites go down

By Brett Wolmarans, Spirent On August 25, 2009
No tags assigned.

Maybe it only seemed like the world stopped for the Michael Jackson memorial on July 7, but there’s one thing we know for sure that did stop.

The City of Los Angeles set up a website to accept donations to offset the $1.4 million required to support the memorial with services like security, traffic control and sanitation. According to CNN, the website crashed repeatedly Tuesday and Wednesday, sometimes for periods as long as 12 hours. Clearly they didn’t test the system to verify it could handle high volumes of traffic. That’s understandable, given the unexpected nature of the event.

During the same time period, a spike in traffic caused by a fare sale at Southwest Airlines resulted in outages and poor performance on their website from 10am to 3pm. The headline read, “Southwest Air Won’t Extend $30 Fare Sale After Web Site Lockup.” Unlike the City of Los Angeles situation, this event was not unexpected. No estimates of the cost of lost business were provided, but five hours of problems during the middle of the business day undoubtedly took a big bite out of revenues, not to mention customer goodwill and corporate reputation.

And that is why test realism matters. Most corporations subject their ecommerce infrastructure to extensive testing before going live. Most likely Southwest did. But it’s very easy to test a system and never find the failure points lurking within. It usually goes like this: set up a test network, hook up some packet blasters and bombard your system with traffic. The results show good performance from the canned traffic and it’s all thumbs up until the system hits the real-world. Then the wheels come off.

Test realism avoids this unfortunate fate by hitting the system with the real-world before deployment. As mentioned in an earlier post [Test Realism], test realism involves:

  • Real user behavior. The flexibility and sophistication to emulate a wide range of user behavior, both benign and malicious.
  • Real converged traffic. The power to create line-rate, fully-emulated, stateful traffic across hundreds of ports.
  • Real network conditions. The power and complexity to create the dynamic, time-varying conditions found on deployed, production networks.

In addition, test realism includes a test methodology based on decades of experience with the technology and application. Of course, you need a test system with the power and scalability to create a real-world environment. But it’s possible to have that test system and still create unrealistic tests. That’s why the test system must have expertise built-in, in the form of test wizards that follow best-practices test methodologies, and also available via professional services for more customized testing.

Unfortunately, in the telecommunications world the lack of real-world testing can result in high-visibility, non-career-enhancing failures. It’s nice when your company is making headlines, but this is one headline you don’t want to be in.

Test realism. Don’t leave the lab without it.

UPDATE: And the hits just keep on coming. At end of July, the website for the “Cash for Clunkers” federal rebate program crashed 15 minutes after it went online due to overwhelming demand. It was down for 5 hours. 

comments powered by Disqus
× Spirent.com uses cookies to enhance and streamline your experience. By continuing to browse our site, you are agreeing to the use of cookies.