When Data Centers Collide - A Modern Fable Of A Lost Lunch

By Andrew Armstrong On July 29, 2009
No tags assigned.

Last week I almost did lunch with a friend, an IT guy. We picked a new deli close to his office and, after a long wait, I finally had a nice Reuben all alone. I texted him and he texted back with apologies and tales of application performance issues.

Being a nice guy, I got a sandwich and chips to go and stopped by his office. I knew I would be welcome. While it might be prudent to beware Greeks bearing gifts most people are glad to see geeks bearing food. Especially when they had to work through lunch because a VP is breathing down their neck about their CRM slowing to a crawl.

My buddy took a short break and told me his sad tale of woe between bites. “We’ve been having issues with our customer relationship management system ever since we installed it. We thought we had it straightened out and picked up on the data center consolidation project.” He paused to wolf down some more sandwich and continued. “Last night we did the cutover from our various corporate data centers to a central facility in Denver. This morning the phones lit up like a radio talk show switchboard. Response times went through the roof. People going to get coffee between screens. I’m not talking down to the break room. I’m talking the Starbucks on the corner.”

I nodded in sympathy and offered him another potato chip.

His situation is not as unusual as it should be. Companies are developing and deploying distributed applications all the time without testing them against realistic WAN conditions. I’ve seen lots of reasons why this happens.

The test plan overlooks the problems that can be caused by a real network. Or they run a few test trials over their production network, usually after hours to prevent business disruptions, and have a false sense of confidence when they don’t see any problems. Or they don’t realize that actual network conditions can be recreated in the test lab.

It’s called network emulation, one of the essential elements of Test Realism.

Test realism lets you do to your application what the WAN is going to do to it, before you have hundreds of users like Kim complaining about your system. The alternative can be ugly. Productivity loss. Finger pointing between the network group and the applications group. Acrimonious meetings. Loss of confidence in your department. Morale issues among your staff.

It’s a shame, really, when, like karaoke, it’s completely preventable.

It is possible to know in advance how your system works on a real network. To test during the relatively calm and rational time of development and testing instead of the high-visibility, company-wide exposure of a post-deployment crisis. To troubleshoot and identify performance and response-time issues early, when it is less expensive to find solutions and implement them.

In fact, it’s not only possible, it’s pretty much essential.

Using test realism in the form of real-world network conditions makes everyday life better for hundreds of thousands, if not millions of people. Kind of like buying a sandwich for everyone.

comments powered by Disqus
× Spirent.com uses cookies to enhance and streamline your experience. By continuing to browse our site, you are agreeing to the use of cookies.