Jump to content, skipping navigation

Avalanche Load Generation White Paper

    * Required Field

    Cancel

    Avalanche Load Generation: How to Improve your Rate-based Tests Product Feature White Paper April, 2005 John Kenney Abstract By allowing you to simulate interactions that are as demanding and volatile as what happens in the real world, Avalanche produces truly meaningful test data. This capability carries with it complexity. This white paper explains a) how load is generated, b) why there may be discrepancies between the desired load and actual load you test, and c) what you can do to eliminate these issues and take full advantage of Avalanche’s rich set of customizable tests. About the Author Dr. John Kenney is Director of Engineering for the Avalanche Product Line at Spirent Communications. Instrumental in the development of the Avalanche and Refl ector network testing appliances, he currently heads the design team working on the continuing refi nement and quality assurance of the Avalanche product line. A ten-year industry veteran, John served as Chief Technology Offi cer and Vice President of Engineering at Caw Networks, and was one of the founders of ePatterns, where he served as both Director of Engineering and Director of Marketing. His distinguished background at Stanford University is highlighed by nine years as a research associate and research assistant, a Master’s degree in Computer Science, and a doctorate in Electrical Engineering Spirent Communications 26750 Agoura Road Calabasas Hills, CA 91302 USA e: enterprise@spirentcom.com Sales Contacts: North America +1 800-927-2660 Europe, Middle East, Africa +33-1-6137-2250 Asia Pacific +852-2166-8382 All Other Regions +1 818-676-2683 www.spirentcom.com/enterprise Copyright 2005 by Spirent Communications, Inc. All rights reserved. Table of Contents 1. Introduction ........................................................................................................................... 1 1.1 Some Defi nitions ...........................................................................................................1 1.2 Traffi c Generation ......................................................................................................... 3 1.3 The User’s Perspective ................................................................................................. 4 2. Problems with Consistency ..................................................................................................4 2.1 Reporting ...................................................................................................................... 5 2.2 Physical Speed and Capacity ....................................................................................... 5 2.3 Animating Users ........................................................................................................... 6 2.4 Graphical Representation ............................................................................................ 6 3. Understanding the Problem Mathematically ........................................................................7 4. Tips and Tricks .....................................................................................................................9 4.1 Planning with Limits in Mind ......................................................................................10 4.2 Areas to Troubleshoot ................................................................................................10 4.2.1 User Profi le ........................................................................................................10 4.2.2 Load Profi le ...................................................................................................... 11 4.2.3 Action Lists ....................................................................................................... 11 5. Conclusion ..........................................................................................................................12 Spirent Communications | Avalanche Load Generation Product Feature White Paper | April 2005 1 1. Introduction It could be Cisco testing a fi rewall or Costco.com testing their Web site: companies who want dependable quality assurance use Spirent. Instead of just blasting devices with traffi c, Spirent’s load generation algorithms allow you to tailor your tests to refl ect what users are really going to put your devices through. Being able to mimic real-world situations provides the most sophisticated— and the most useful—information about how your equipment or Web site performs. 1 This white paper explains how Avalanche generates load, why variations in the generated load exist, and what tricks of the trade you can use to improve your rate-based tests. 1.1 Some Defi nitions In Illustration 1, the user has decided to test how a fi rewall handles 20,000 TCP connections. This type of measurement is known as a concurrent metric because the 20,000 connections are all hap- pening at the same time and new connections only open to replace connections that have closed. In other words, the number of connections will stay constant for the duration of the test. 2 Other examples of concurrent metrics would be testing a set number of simulated users or HTTP transactions. Illustration 1: Concurrent Metrics Testing 20,000 Connections 1 This white paper’s focus is on realistic testing, but Spirent’s tools allow you to isolate and verify faults with unrealistic tests, too. These tests are predictable, repeatable, and laboratory-precise. 2 In real life, connections close for a variety of reasons (for example, if a Web page has fi nished downloading). You can keep connections open indefi nitely if you are performing unrealistic tests. Open Connections Capacity Test — 20,000 Open Connections 20,000 16,000 12,000 8,000 4,000 0 OP EN CONNECTIONS TIME 00:04:00 00:04:20 00:04:40 00:05:00 00:05:20 Desired Current Spirent Communications | Avalanche Load Generation Product Feature White Paper | April 2005 2 Illustration 2 tells a different story. It demonstrates a rate-based metric—in this case, the user’s test is to add 20,000 connections each second. Avalanche works to keep these rates accurate, but there are circumstances under which Avalanche seems to add 19,900 connections/sec or 20,100 connec- tions/sec instead. In Section 3, we’ll be describing more about why it isn’t possible to always add exactly 20,000 connections/sec. We’ll also be giving some tips to get very close. 3 Illustration 2: Rate-based Metrics Testing 20,000 Connections/Sec To understand the difference between concurrent metrics and rate-based metrics, imagine load test- ing a restaurant. If we want to stress test Mel’s Diner, we could see when failure occurs if there are always 100 customers. This would be a concurrent metric—people come in to eat, fi nish and leave, but new customers immediately take their places, keeping the restaurant packed at a constant 100. Maybe it takes two days before the kitchen overheats and the cooks collapse, maybe it takes two hours. Before any catastrophic failure, a stress test will usually fi rst generate a slow-down. Mel’s Diner could handle 100 customers for a while, but food would take progressively longer to arrive and the quality would probably deteriorate. We could also run a stress test with a rate-based metric. This kind of test would start Mel’s Diner off at 6 a.m. with zero customers. Every hour would bring 30 customers (in other words, one customer every two minutes). With every passing moment, let’s assume there are more customers arriving than there are custom- ers fi nishing up and leaving. As time goes on, the diner gets overloaded; the number of customers waiting for service increases and the time each customer spends waiting also increases. 3 Please note that “load specifi cation” is really just another name for these metrics. 20,000 16,000 12,000 8,000 4,000 0 CONNECTIONS/SEC TIME 00:04:00 00:04:20 00:04:40 00:05:00 00:05:20 Desired Current Connections/Sec Rate Test — 20,000 Connections/Sec Spirent Communications | Avalanche Load Generation Product Feature White Paper | April 2005 3 Yet more customers keep crowding in each minute. By mid-afternoon there are so many customers waiting that the waiters can’t move and no one eats—Mel’s Diner has reached its failure point. 4 1.2 Traffi c Generation In approaching the problem of simulating real-world traffi c, Spirent engineers had to contend with a seeming contradiction. On the one hand, real-world traffi c is unpredictable and bursty; on the other, a good testing tool is predictable and smooth. Avalanche solves this predicament through the use of a Load Controller and a Load Engine. Measurements, often based on the state machines of the various protocols in the traffi c, are fed into the Load Controller. The Load Controller is in charge of starting and stopping traffi c, as well as adding and removing it. The instructions from the Load Controller go into the Load Engine. It’s the Load Engine that generates the traffi c. The Load Engine also passes feedback measurements to the Load Controller. Illustration 3: The Load Generation Architecture The feedback cycle between the Load Controller and the Load Engine (Illustration 3) makes Ava- lanche a self-regulating traffi c system. It’s important to realize that the Load Controller keeps a his- tory of measurements and uses that history to determine how much traffi c to add. Though the load is generated very quickly, the system must take into account that it does take some time to push out the traffi c. Similarly, it does take some small fraction of time to look at the measurements. 4 However, the performance of the diner degraded before it completely failed—customers had to wait increasingly long times before being served. Spirent Communications | Avalanche Load Generation Product Feature White Paper | April 2005 4 Though the combination of this lag is on the order of milliseconds, events can happen during this time that will have an impact on the consistency of the generated load. 5 1.3 The User’s Perspective Avalanche generates simulated Internet traffi c that factors in real-world characteristics such as con- nection speed, packet loss, browser emulation, think-time and aborted transactions. Its model for simulation is centered upon the concept of the simulated user, which is a combination of a User Profi le and a corresponding Action List. The User Profi le describes the user’s behavior. For example, User Profi les specify the network characteristics of the users including their link speed, network latency and IP address range. The Action List includes the host name or host address and the “ac- tion” to perform—retrieve a Web page, send an email, play a media fi le, etc. A simulated user is born, walks the Action List (following the behavior indicated by its User Profi le), and dies. All load is generated this way—by creating simulated users, having them perform a specifi c set of tasks and then disappear. To generate more load, the system adds more simulated users. The number of simulated users generated will satisfy whatever type of load you have specifi ed, whether you speci- fi ed users, connections or transactions. One simulated user does not equal a predictable amount of traffi c. Rather, the traffi c generated is heavily dependent upon the performance of the device or system under testing. As the device or system slows down or fails, existing simulated users have a harder time making it through the Action List, so fewer simulated users die. Since fewer simulated users are dying, concurrent metrics will give birth to few new ones. So the traffi c generated is reduced as the device fails. This is a more accurate way to test load than just hitting one URL a million times, since that is not what happens in the real world. That simple type of load testing, however, is far easier to control and attenuate. The key difference with Avalanche is that the atomic metric of load generation is the user session, which plays out over a non-trivial amount of time, thus simulating real-world condi- tions but not allowing instantaneous changes to generated load levels. A realistic test allows you to examine your device or system at work under a variety of condi- tions—not just one or two. Testing a variety of conditions translates into testing scenarios that have a multiple variables—for example, both how fast the Device Under Test (DUT) can initiate a TCP connection, as well as how fast the DUT can process an action from the Action List and return an acknowledgment. Simulated users are at the heart of Avalanche because the user perspective allows tests to accurately synthesize all the independent variables that are involved in realistic testing. 2. Problems with Inconsistency The previous sections have defi ned our basic terms; now that these pieces are in place, we can talk about why sometimes the actual load being tested does not match the desired load that you specifi ed. 5 In addition to measuring traffi c, making load decisions and adding users, the processor is also responsible for generating reports and handling the users that are already in progress. These last two tasks are more processor-intensive than the other three. Since there is usually only a single processor doing all of this work, we can understand the lag by looking at required processor cycles. Spirent Communications | Avalanche Load Generation Product Feature White Paper | April 2005 5 One of your fi rst steps towards eliminating inconsistencies is determining the root of the problem. Some limitations are due to the device or system that you’re testing; others are limitations in Ava- lanche. In addition to the “desired” and “actual” load results, pay attention to Avalanche’s attempts to get to the desired load. When you are testing connections/second, for example, look at the Details tab of your test. In this area you can see attempted connections and actual, established connections. If the number of attempted connections is equal to the number of connections you want to test, then the problem occurs in the establishment of the connections. This suggests that the limitation is in the device or system you’re testing. The rest of this section investigates parts of Avalanche’s load generation that can produce inconsis- tencies. There are four main areas: • Reporting • Physical speed and capacity of the Load Generator • Number of animating users • Graphical representations 2.1 Reporting Avalanche reports its statistics every four seconds. The actual load and the desired load may not appear to match because of changes in the desired load over that reporting interval. For rate-based load testing, the load generator examines the recent history to see how much load has been added; it divides this change in load by the amount of time over which that load was added. If this amount is less than the desired load, the Load Controller will issue an order to add more simulated users. This approach works pretty well, but if you have defi ned a test that changes more often than every four seconds, the load graph that Avalanche produces may not look like the load graph you speci- fi ed. Though Avalanche is fl exible enough to handle these scenarios, there will be a visual incongruity. 2.2 Physical Speed and Capacity When the physical speed of the Load Generator is exhausted, it can no longer meet the desired load. There are limits on how much input/output can be generated based upon the capabilities of the Network Interface Card (NIC). Spirent has several different cards available to help you meet your requirements for testing large numbers of packets/second. Please contact a sales representative for more information. In addition to the NIC, there are limits to CPU processor speeds. Some protocols like HTTPS, RTSP and RTP require signifi cant processing for each action. Depending up the CPU, the number of ac- tions that are started and already underway may be limited. This sort of performance limitation in the CPU is often indicated by an increase in the number of animated users. See Section 2.3 for more details. Spirent Communications | Avalanche Load Generation Product Feature White Paper | April 2005 6 The fi nal physical piece is physical capacity. You can use the “resources” tab of Avalanche to see the amount of memory available (as well as the amount of memory and packet memory used). 6 If the amount of memory used is above 93% of the available, the Load Generator will not be able to add more users since there is insuffi cient memory to allocate to them. As in-process simulated users die, more memory is freed up. But rate-based metrics call for an ever increasing number of simulated users, so the few that die cannot free up enough memory for all of the new ones that the Load Generator wants to add. Load generation will continue at approximately the maximum that the appliance can generate—for better results, lower the amount of load you have specifi ed and run the test again. 2.3 Animating Users Simulated users that are actively traversing their Action List are called animating users. In rate- based metrics, the number of animating users may increase too quickly. Creating and maintaining users involves a fair number of processor cycles. The amount of time it takes to handle the work-in- progress infl uences (and restricts) the Load Generator’s ability to measure and add more load. Transmission errors on the wire and the limitations of the device under testing can both create situations where users cannot process the entire Action List and so get stuck mid-way through their life-cycle, delayed by retransmissions and timeouts. These users do not die because they haven’t completed their work, but their existence hampers additional rate-based load. To determine whether there is a problem with animating users, check the SimUsers bubble graph tab of the Statistics Details. If living and animating users account for more than 1% of the desired rate-based load, there is most likely a problem; you will also observe an ever increasing number of animating users. If this is happening in a rate-based test, the Load Generator will probably not be able to keep up. Reduce the amount of desired load and re-run the test. 2.4 Graphical Representation When graphs are based upon statistics, it seems reasonable enough to conclude that they are based on facts. But statistics are slippery—as much art as they are science, their interpretations should be more suspect than they usually are. One of the most common misuses of statistics is the “gee-whiz” graph, which inserts nonexistent drama into statistical trends. These can be purposefully constructed, but it’s easy to amplify trends accidentally, too. A huge discrepancy will look minor if the graph is zoomed out far enough. Like- wise (and more relevant to our problem), a 1% error will rise precipitously as someone zooms in and may even look like a signifi cant and imposing inconsistency. 7 The base for your graph can be something other than zero—check to make sure the traffi c it describes is what you expect it to describe and that you are not too far zoomed in or out. 6 Avalanche reports this information in real-time as tests are run. You can view physical capacity and other data by looking in the statistics details of a report that is underway. 7 To read more about the implications of this, see How to Lie with Statistics by Darrell Huff. Spirent Communications | Avalanche Load Generation Product Feature White Paper | April 2005 7 3. Understanding the Problem Mathematically Let’s revisit Illustrations 1 and 2. In Illustration 1, you have set up your load specifi cation to main- tain 20,000 open connections. Maybe you know that your equipment can handle 20,000 connec- tions, but you want to see if it can run at that level for 30 days. During this stress test, connections will open and close, but because the only task is to keep 20,000 connections open, the Load Con- troller simply tells the Load Engine that it should add a connection for each connection it removes. It’s very clear-cut to maintain 20,000 connections. There is no impact from the time to look at traffi c or time to generate it. The fi rst part of the concurrent metrics algorithm dictates how to generate load and continue the traffi c already generated. This may be followed by a measure and report cycle, which in turn may be followed by a measure and add load cycle. The work-in-progress may have resulted in the death of many simulated users, so the load number reported may be temporarily below what is desired— however, the next cycle will begin with the addition of new users to get the load back up to the target amount. The algorithm for rate-based load metrics is similar. The exception is that the Load Controller uses additional processor cycles to look at the history of measurements—this information overwhelm- ingly determines how much load should be added. Back in Illustration 2, the system is trying to maintain a connection rate of 20,000 connections per second, while some number of connections, are closing. Since we know that it takes us time to look at the history and time to generate the traffi c, we can add an adjustment factor that will offset these numbers. This is what the Load Controller does. The performance coeffi cient in the equation is a dynamic, well-educated approximation. The value will be close but not exact. For example, illustration 4, shows this variation. At t=1.8 we’ve added an additional 18,000 connections and we’re on track. By t=1.9, we’ve added a further 1,000 connections. So far, so good. The Load Engine sends measurements back to the Load Controller and says we need another 1,000 connec- tions over the next .1 seconds. It has taken time for this measurement to get to the Load Controller and the Load Controller knows this. The Load Controller also knows that it will take a small amount of time to generate the traffi c once it sends the command back to the Load Engine. The performance coeffi cient takes all of this into consideration and is part of the actual algorithm. With the performance coeffi cient in place, the algorithms can predict that since it’s going to take more time than just .1 seconds, the Load Engine should add 1,200 connections. Every device and system under test is different. The performance coeffi cient is part of a complex algorithm to get this right, but it’s possible that it could be slightly off. So by the time the Load Controller measures again at t=2.1, it is possible we could still be 100 connections below even though we added those 200 extra connections at t=1.9. 8 8 The Load Controller’s algorithm will determine how many new connections to add in a current time inter- val based upon a) the requested number of connections per second, b) how long the tested system is taking to accept new connections, and c) the amount of time until the Load Controller next monitors connection progress. Spirent Communications | Avalanche Load Generation Product Feature White Paper | April 2005 8 This is a 0.5% error. If the scale of the x- and y-axes is large, this doesn’t look like too much of a problem, but when you zoom in on the graph, this 0.5% error looks a lot bigger. Illustration 4: Our Rate-based Test Zoomed in to t=1.8 Part of the problem here has to do with the fact that connections are discrete objects—there is no such thing as half a connection or half of a simulated user. The other part has to do with the fact that anything that measures a rate of change does so by approximation. If we take any point on the curve of a rate of change, we get an instantaneous look at the rate of change—but this is always an approximation. 9 Even an instantaneous measurement requires some change in time and this change in time can have a huge impact when discrete math is used. So while calculus gives us the tools to tackle this problem, it also limits us from solving it completely. The laws of calculus imply that the smoothness of the rate of change of time-based metrics (such as connection/sec or users/sec) is determined by the interval of the time being used in the calcula- tion. The smoothest graphs and most accurate measurements are determined when the smallest possible time differences are used (i.e. as delta t approaches zero for the derivative value). The timing of Avalanche’s simulation is determined by discrete values such as connection establish- ment, latency and processing time; in real-world scenarios, these metrics can never be reduced down to zero. Because of this, the rate of change in the number of connections over time will never be as smooth as a theoretical model would allow, and changes to the number of users or connections can never be made instantaneously. Even if the time interval of calculation and response could be reduced down to near zero, calculus states that no change in the number of users or connections 9 Rate-based metrics like connections/sec are derivatives. 20,100 20,000 19,900 0 CONNECTIONS/SEC TIME 0 1.8 2.0 Desired Current Connections/Sec Rate Test — 20,000 Connections/Sec Spirent Communications | Avalanche Load Generation Product Feature White Paper | April 2005 9 is possible unless some amount of time has passed. The real-world conditions upon which Ava- lanche bases its testing mathematically limit how nimble it can be when making adjustments to these time-based values. To return to the example of Mel’s Diner, consider a graph charting restaurant capacity over time. A major component of the line for this graph is determined by the amount of time it takes for a typi- cal diner to fi nish his meal. If everybody took 10 seconds to sit, eat, and leave, the restaurant could make very fast adjustments to the number of people being served at a particular time. But long din- ing time means more lag time in adjustment, and aberrant situations, such as that guy who takes 30 minutes to fi nish his dessert, introduce a variability that cannot be easily handled. 4. Tips and Tricks There are still occasions when your actual rate may be more than just a small blip away from your desired rate. There are a number of methods that will reduce these more dramatic differences. Let’s say you’ve set up your test to add 250 connections/sec, but when you look at what is hap- pening you see a large discrepancy between your desired rate and the actual one. The graph below starts at minute 3:10. Illustration 5: A Discrepancy Between Desired Load and Actual Load This section will take you through a series of troubleshooting techniques to get your actual load to correspond to your desired load. 4.1 Planning with Limits in Mind The fi rst thing to think about when you’re planning a test is what your goal is—if you just want to test the bandwidth in a particular protocol, the simpler you can make the test, the better the infor- mation you’re going to get about how the device is performing. 400 300 200 100 0 CONNECTIONS/SEC TIME 00:03:30 00:03:50 00:04:10 00:04:30 00:04:50 00:05:10 Desired Current Connections/Sec Spirent Communications | Avalanche Load Generation Product Feature White Paper | April 2005 10 At other times, you would like to get bandwidth information, but also want a certain number of connections open and some new connections added each second. Under most circumstances, Avalanche can handle these complex testing requirements; however, there are physical limits. You shouldn’t expect to test the upper limits of bandwidth with Avalanche if you’re pulling a large object and opening an enormous number of connections/second at the same time. If you are encountering problems, it may be because your goals are too grand. Try testing capabilities separately. Customer support may be able to provide you with benchmarks for the particular hardware and software you are testing. The general rule of thumb is that bandwidth equals the number of connections per second times the object size. bw = connections/sec * object_size 4.2 Areas to Troubleshoot Every test is composed of three main components—the User Profi le, the Load Profi le and the Action List. The following suggestions target these places. 4.2.1 User Profi le The User Profi le mimics a real user’s behavior, so it can affect the amount of load that you can effectively test. Some user profi le settings make each simulated user do more work, which means fewer resources are devoted to creating simulated users and more resources can be directed to test- ing connections or transactions. The main infl uence is Think Time—the amount of time that the simulated users will spend in a giv- en process (for example, how long they will take to browse one Web page before clicking through to another). Increasing the Think Time for concurrent tests allows you to test a higher load. From Avalanche’s perspective, Think Time is really idle time for that simulated user, so it gives Avalanche time to do something else for other users. This time can be used to generate additional users or to add new connections. For rate-based metrics, however, longer think times decrease performance. A longer Think Time means you need to create (and manage) more simulated users to achieve the desired rate—and each simulated user does less work. Increasing the Think Time for rate-based metrics, like connec- tions/second or bandwidth, lengthens internal queues and makes more intensive demands on the processor. If you are testing HTTP, also consider the impact of the persistence feature of HTTP 1.1. The persis- tence feature of HTTP 1.1 means that connections are shared. When your simulated users use HTTP 1.1, they will open one connection and be able to issue many requests through that single connec- tion. The result is that you can test higher transactions per second by using HTTP 1.1. 4.2.2 Load Profi le A load profi le allows you to dictate how quickly to ramp up to your target rate and how to step the rate up thereafter. It is possible to describe a load profi le with a duration that is too short to Spirent Communications | Avalanche Load Generation Product Feature White Paper | April 2005 11 actually carry out the amount of work specifi ed in the workfl ow. Take, for example, Illustration 6. At fi rst glance, there appears to be a problem—the test continues to generate load after 2 minutes, even though the desired load (blue) should be zero. The current load (green), continues to stair- step down the number of connections being tested. Illustration 6: Why Isn’t the Load Decreasing? It turns out that this is not actually a problem of the system, but one of workload. If the size of the objects being pulled is too big, Avalanche will keep the connections open as long as it needs to in order to fi nish—even if that goes past the time set in the duration portion of the load profi le. After we decrease the size of the object (or lengthen the duration), the test will end as specifi ed. 4.2.3 Action Lists When you are testing bandwidth or connections/second, Spirent recommends that you place be- tween four and 12 entries in your Action List (when you are testing a Web site, you can test howev- er many entries are appropriate). If you have only one entry in your Action List, too many simulated users may be created—remember that after a simulated user travels to each URL, it dies and a new one is born: short lists mean too many simulated users. 10 At the other extreme, too many entries on an Action List may reduce the effi ciency with which the Load Engine creates new simulated users. 11 The entries in the Action List are really objects that get pulled across as simulated users encounter them. When taken with bandwidth and the number 10 There is overhead associated with the creation of a simulated user’s and for HTTP, keep-alive and non- keep-alive connections affect performance. Connections 20,000 18,000 16,000 12,000 8,000 4,000 0 OP EN CONNECTIONS TIME Desired Current 00:00:40 00:01:00 00:01:20 00:01:40 00:02:00 00:02:20 Spirent Communications | Avalanche Load Generation Product Feature White Paper | April 2005 12 of connections, large objects can trigger the desired-actual discrepancy we’ve been addressing. In general, keep in mind the equation bandwidth = connections * object_size when you are creating your tests. The upper limit for these variables is determined by the bandwidth capacity of the network and server devices being tested, among other things. The fi nal piece to watch with Action Lists is mixing protocols. If you are testing transactions or transactions/second, please note that your Action List should only involve HTTP and HTTPS. For other protocols, such as FTP, DNS, Streaming, POP3, SMTP, you should use the simulated user or simulated user/second specifi cation—transactions are strictly for HTTP/HTTPS. The table below pro- vides a summary of which tests can be run for which protocols. Simulated user-based Connection- based Transaction- based Bandwidth Body byte 12 HTTP/HTTPS X X X X X FTP X X X DNS X X Others X For all TCP-based protocols X Table 1: Recommended Tests for Various Protocols 5. Conclusion In this paper we’ve reviewed how load generation works and given some helpful hints on how to take advantage of Avalanche’s rich set of customizable tests. Spirent’s algorithms give you an unprecedented amount of control. By allowing you to simulate interactions that are as demanding and volatile as what happens in the real world, Avalanche also produces truly meaningful test data—numbers you can count on. These capabilities carry with them some complexity, but Avalanche remains the most usable, reliable and robust testing tool available. 11 This is because the Load Engine looks ahead to fi gure out how many simulated users to create; if the list is too long, the Load Engine may fail to add a suffi cient number of simulated users at the right times. There are two issues here: fi rst, it is harder to control rate-based loads that test transactions/second, and second, the Load Controller may get called less frequently, making its estimates less granular. 12 Body byte metrics should be run with rate-based constraints. ©2005 Spirent Communications. All rights reserved. Spirent Communications, the Spirent Communications logo, Avalanche SMB, Smartbits 6000B, Refl ector 2500, Refl ector 220, Avalanche 2500, and Avalanche 220 are trademarks of Spirent Communications.