Jump to content, skipping navigation

Using Spirent TestCenter to Generate Real World Traffic Whitepaper

    * Required Field

    Cancel

    White Paper Using Spirent TestCenter™ to Generate Real-World Traffic by Angus Ma January 2008 i Contents Overview..................................................................................................... 1 Evolution of Test Platforms............................................................................... 2 First Generation (Traditional packet generators and analyzers) ...........................................2 Second Generation (SmartBits and other network testers)..................................................2 Third Generation (Spirent TestCenter) .........................................................................3 SmartBits vs. Spirent TestCenter ........................................................................ 4 Monolithic Traffic Model vs. Object-Oriented Model .........................................................5 Layered Architecture ..............................................................................................7 Sample Test Scenarios .............................................................................................8 Generating Bursty Traffic .........................................................................................9 Creating Host and Stream Blocks .............................................................................. 10 Traffic Policing.................................................................................................... 13 Traffic Prioritization ............................................................................................. 13 Varying Traffic Load Automatically............................................................................ 14 QoS Simulation .................................................................................................... 15 First Level Analysis – Counter Subscriptions.................................................................. 16 Second Level Analysis – Template Filter Analysis ........................................................... 16 Third Level Analysis – Real-Time Capture .................................................................... 16 Conclusion ..................................................................................................17 White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 1 - January 2008 Overview Test instruments have undergone evolutionary change in the last 15 years. In the past few years, however, test labs have come under intense pressure to extensively test products and networks using numerous test cases and “real-world” traffic. Moreover, these tests must be accomplished against highly complex DUTs (Devices Under Test) using shorter test cycles. SmartBits and other testers of the same generation ran into realism limitations inherent in hardware components and the software model of their day. To solve such issues, Spirent Communications developed a new approach. Spirent TestCenter replaces SmartBits and other testers by incorporating fundamentally improved architectural changes. For example, Spirent TestCenter offers an object-oriented model for defining traffic utilizing the advantages of encapsulation and inheritance. The tester also offers cutting edge hardware components to enable complexity and real-time control. These changes address the industry’s current test needs by providing the following: • Independent scheduling of packets from different traffic sources for each port • Mixing of bursty (e.g. VBR) and steady-state (e.g. CBR) traffic on the same port at the same time • Measure of true jitter latency • Independent global changes of traffic flows since the attributes are now associated with objects such as host blocks and stream blocks • Many changes taking effect without having to stop and restart a test • Inheritance and encapsulation that speed up test time and reduce error by allowing attributes to flow up and down the stack as in a real network • Spirent TestCenter keeps the state of the tests in the test microkernel BLL (Business Logic Layer), and the presentation layer is kept in both the GUI and API • Universal, object-oriented API simplifies testing and reduces error • Up to 100X improvement in test time is achieved These changes make it practicable to test networks more thoroughly under complex but realistic traffic conditions. Easier global changes mean more extensive testing is possible, which will make products and networks more reliable and predictable while improving realism. White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 2 - January 2008 Evolution of Test Platforms Traffic generators and other test instruments have been part of the standard toolbox in test labs of network equipment manufacturers (NEMs), service providers (SPs) and large enterprises for the past 15 years. As the complexity of networks and network equipment increases, the functionality of the test platform must keep pace in hardware, tool sophistication and usability. Since the 1990s, network testing has undergone the following major evolutionary changes: First Generation (Traditional packet generators and analyzers) �ƒ Generates packets at wire speed using a single and later multiple ports; single stream or “Traditional Mode” �ƒ No concept of “Streams” or “Measured Pathways” �ƒ Aimed at benchmarking routers and switches using RFCs 2544 and 2889 �ƒ Key metrics measured: throughput, latency and frame loss �ƒ Traffic is homogeneous – all frames are identical �ƒ Target DUT is a Layer 2 switch Second Generation (SmartBits and other network testers) �ƒ Focus on QoS; added “Streams” or “Measured Pathways” �ƒ Aimed at measuring how different traffic types at different code points are handled by the DUT �ƒ Added state machine management (BGP, OSPF, IS-IS) �ƒ Measures metrics for each traffic flow �ƒ Batch oriented system �ƒ Traffic pattern is port specific – i.e., all flows from the same port have the same attributes �ƒ Target DUTs: Layer 2 and Layer 3 switches as well as routers White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 3 - January 2008 Third Generation (Spirent TestCenter) �ƒ Simulates large networks and “real-world” traffic in real time �ƒ Object-oriented just like production networks �ƒ Realism oriented; layers 2-7, complex managed network transports �ƒ Focus on scalability and test automation �ƒ Supports inheritance and encapsulation for reduction in test error, improved test time and future proofing of the test model �ƒ Generates traffic from a large number of hosts from each test port �ƒ Automated, user-defined result analysis The following diagram illustrates the three generations of test instrument: White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 4 - January 2008 SmartBits vs. Spirent TestCenter SmartBits is an example of a second-generation test platform. It was extensively used in the late 1990s and early 2000s to test routers and switches. In addition to benchmarking Layer 2 and Layer 3 switches, SmartBits also performed simple QoS testing. For example, a SmartMetrics card can simulate up to 1000 flows, tracking multiple metrics per “pathway.” A packet signature is included with each frame, allowing the receiving port to track the order and latency of the traffic for each flow. In the past few years, greater emphasis has been placed on simulating large-scale networks with realistic traffic patterns (increasing CAM density, route table density, new services, etc.). In running through the new test cases, SmartBits platform encountered limitations inherent in the chassis and controlling software: �ƒ Creating and editing the traffic flows is a time-consuming and error-prone process �ƒ There is no easy way to make global changes (e.g. changing the source and destination addresses or the default gateway of flows) other than manually �ƒ Scheduling is done on a per-port basis, and the traffic pattern of each port is fixed. For example, if a port has 10 flows, each flow generates traffic at the same rate (e.g. constant bit rate or bursty) �ƒ Apart from a few standard tests, there is no easy way to automate test cases In 2006, Spirent introduced its third generation test platform, the Spirent TestCenter, which replaces the SmartBits and other testers. Apart from using more powerful hardware such as next generation FPGA and processor technology, the new platform incorporates two major architectural improvements that significantly enhance its capabilities – an object-oriented model and a layered architecture to be discussed in an upcoming section. To compare the power difference, the SmartBits with the SmartMetrics card generates up to 1,000 flows per card while the new Spirent TestCenter generates either 16,000 or 32,000 flows per port depending on card used. White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 5 - January 2008 Monolithic Traffic Model vs. Object-Oriented Model The SmartBits platform uses a monolithic model for traffic generation. One creates a flow by specifying the source and destination addresses, frame type, prototype field and contents of the packet. To create a large number of flows, the system has to maintain a table containing information about each flow. To change all the flows to use TCP port X instead of Y, one must edit each flow and make the change accordingly – a very time-consuming and error-prone procedure. As the test becomes more complex, the user must manually manage tabular relationships because all tables are in the same scope. The following diagram illustrates the monolithic model used in SmartBits and other testers. In contrast to SmartBits, the Spirent TestCenter platform operates on abstract objects such as hosts, streams and routers, and the relationships between them. For example, a test port can simulate a number of hosts, each of which can either be a single host or a block of hosts. Once hosts have been defined, streams can be constructed between these hosts. White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 6 - January 2008 In the following example, hosts 1 and 3 are defined as single entities whereas hosts 2 and 4 represent host blocks that simulate a number of devices each with a unique address. Similarly, stream 1 is a simple traffic flow that carries packets from host 1 to host 3 whereas stream 2 is a stream block that contains multiple traffic flows between hosts 2 and 4. The concept of the stream block is very powerful. For example, if “host 2” is increased from 10 hosts to 100 hosts, stream 2 will be changed to 100 flows automatically since it uses host 2 as the source. Furthermore, if stream 2 is reconfigured to use a different TCP port, then all associated flows will follow as well. With an older test platform, such changes entail extensive manual editing. Since these changes are propagated automatically through the objects in Spirent TestCenter, another benefit is that for the changes to take effect, it is no longer necessary to stop and restart the test. This saves a large amount of time during testing. White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 7 - January 2008 Layered Architecture As mentioned earlier, the SmartBits and other testers’ architectures are monolithic. This means the user interface and underlying tests are tightly coupled. The user interface must be reworked to accommodate any changes in the tests. Spirent TestCenter has adopted a layered architecture. The software can be divided into three distinct layers: �ƒ Viewer – This layer does not contain the state of the running test. Rather, it is only responsible for interacting with the test and the presentation of the data. For example, there is a GUI viewer and a viewer for each API environment. For example, the TCL viewer allows a TCL script to configure and initiate tests and to retrieve the results. �ƒ BLL (Business Logic Layer) – This layer is equivalent to the kernel of an operating system and performs all housekeeping functions. In particular, it keeps track of the state of the test including objects and counters. The API is all implemented in the BLL. �ƒ IL (Instrumentation Layer) – This layer interacts with the hardware in the test platform. Theoretically, the BLL could run over a different IL which would control a test platform other than the Spirent TestCenter. To the user, the main benefit of this layered architecture is any changes to the underlying tests (in the BLL) will be immediately reflected in the viewer. There is no time lag between the introduction of a new test and availability of the API to TCL, Perl or Python. White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 8 - January 2008 Sample Test Scenarios The following are examples of tests that the author ran on the Spirent TestCenter platform which would be impossible or very difficult to perform using the SmartBits. These examples illustrate that more detailed and extensive test cases are now possible with a third generation test platform. The key objective of these tests was to characterize the performance of a QoS-enabled provider network under different traffic conditions. The following is a diagram of a typical service provider network. In this type of network, the ingress traffic into the PE router consists of the aggregate traffic generated by all the hosts within the customer network. It may have a mix of video, voice and data traffic tagged appropriately. The challenge is how can traffic be generated that adequately simulates the real-life traffic as seen by the PE router? White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 9 - January 2008 Generating Bursty Traffic Using the Stream Block Editor, the traffic load can be specified. The following screen shot shows a traffic pattern of four 1000-byte frames every 4 seconds (4000 milliseconds). The following shows the resulting traffic rate which alternated between 0 and 32 Kbps (4 frames x 1,000 bytes x 8 bits) every 4 seconds. White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 10 - January 2008 With the object-oriented model, the stream attributes can be modified and the changes take effect without stopping and restarting the test. The following shows the effect of reducing the inter-burst gap from 4 seconds to 2 seconds. Creating Host and Stream Blocks A host block allows a host to simulate a large number of sources and destinations. The above configuration defines 100 hosts, each with a unique MAC and IPv4 address. These hosts will occupy the address space from 172.18.0.100 to 172.18.0.199. Note they all use the same default gateway at 172.18.0.1. If this needs to be changed to point to a different router, all simulated hosts will be updated at the same time. White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 11 - January 2008 A similar host block was defined on the destination port. A stream block was then defined connecting the two: The result is 100 flows between the two host blocks: 172.18.0.100 -> 172.19.0.100 172.18.0.101 -> 172.19.0.101 172.18.0.102 -> 172.19.0.102 … 172.18.0.199 -> 172.19.0.199 As mentioned earlier, if the host block is changed to 1,000 hosts, the same stream block will be updated automatically to have 1,000 flows. White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 12 - January 2008 In the following test, each flow was configured to transmit at a constant rate of 8 Kbps. The following shows the result when the constant bit rate traffic was multiplexed with the bursty traffic: This process can be repeated to generate an aggregate of an arbitrary number of independent traffic flows. For example, the following test had three independent types of traffic coming from a single port: �ƒ Constant bit rate traffic generated by a host block �ƒ Bursty traffic with an inter-burst gap of 20 seconds �ƒ Bursty traffic with an inter-burst gap of 5 seconds White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 13 - January 2008 Traffic Policing In this test, each flow was transmitting at a rate of around 170 Kbps. Traffic policing was enabled, throttling the traffic of one stream down to 100 Kbps. The following shows the result: Traffic Prioritization In the test, traffic from one flow was given higher priority (e.g. real-time traffic). When there was no contention, all flows experienced similar latencies. Traffic policing Same latency for all flows when there is no contention White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 14 - January 2008 However, as the traffic of each flow was gradually increased (without stopping and restarting tests), there was contention and the high-priority traffic experienced shorter delays. Varying Traffic Load Automatically By combining the command sequencer, another new feature in the Spirent TestCenter, traffic load can be varied automatically during the execution of a test case. The command sequencer uses a drag-and-drop user interface that allows a user to specify commands to execute automatically. Furthermore, each command can be tailored using custom parameters. For example, the following sequence will start the traffic generator, run for 30 seconds and then change the load iteratively. Low latency for real-time traffic White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 15 - January 2008 The load variation can be configured through the parameters of the “Iterate Load Size” command. For example, using the Inter Burst Gap as the unit of change, the load can be varied by specifing the start gap, stop gap and the increment during each iteration. This adds another level of flexibility during testing. QoS Simulation Besides traditional emulation of QoS routers and routing protocols, Spirent TestCenter Port Priority Mode requeues traffic according to the global QoS traffic policy. This is a very significant advancement in QoS testing because for the first time, not only can test equipment mark DSCP, TOS, and 802.1p code points but can also reshape traffic just like a real device. This ability, through the port scheduler, is first implemented through “Port Priority Mode.” This mode implements strict requeuing based on a policy. Thus, the user can clone the QoS policy on the DUT to the emulated routers and have the test equipment reshape traffic in real-time. For example, the user can now create background best effort traffic with an IMIX (Internet Mix) pattern. Then, in real-time, set up SIP VoIP traffic, mark it as “real-time control” and mix it in. The emulated router will make room for the higher priority traffic by delaying the lower priority background traffic. From a testing perspective, now the user can directly test how the DUT handles variable QoS traffic by causing more queuing exceptions and forcing an ASIC or CPU to process the QoS flow as opposed to the hardware CAM. Since the packet scheduler is a “plug-In,” other QoS scheduling schemes such as WRED (Weighted Random Early Discard) can also be created and added. White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 16 - January 2008 First Level Analysis – Counter Subscriptions Spirent TestCenter simultaneously measures up to 41 performance metrics. Further, one never loses granularity as the stream density increases, unlike other testers. To simplify the counter view for the user, the variables displayed can be tuned. If the test does not involve IPv6, the user does not need to count or display these variables. By subscribing to certain variables, the user can see just what is relevant to the test. Because the user can create his own variables into views, and switch views on-the-fly, the user has unprecedented flexibility in counting. Second Level Analysis – Template Filter Analysis Traditional analysis using first and second generation tools required users to spend extra time analyzing traffic after a test. If a test generated large data sets, the user had to manually parse the data, a process that could take hours. Spirent TestCenter revolutionized real-time analysis using template filters. These filters, similar to Microsoft® Excel® pivot tables, allow users in real-time to simply pivot the data set to the variable of most interest. For example, a user creates a test containing 10,000 streams and 4095 VLANs. By default, the user sees performance per stream. Now, with a few clicks, the user can see performance “by VLAN.” Third Level Analysis – Real-Time Capture Sometimes the test requires the user to inspect the data stream directly. Spirent TestCenter offers real-time capture and analysis. The user is given the option of setting up hardware- based filters critical for high speed interfaces or creating “capture rules;” for example, capture all traffic from a specific source, traffic that has a specific pattern or traffic with errors. This “capture policy” can be applied to many ports. Once traffic flows into the decoder, the data stream is displayed in real-time including direct decoding of the performance signature field. This utility will then analyze the captured data and generate a report. If the decoder detects well known protocols such as VoIP or video, the tool will also extract the presentation layer from the data stream and reconstruct the traffic, allowing the user to listen to the voice or watch the video. White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 17 - January 2008 Conclusion The Spirent TestCenter platform, with its object oriented approach and layered architecture, provides a number significant advantages to testers vis-à-vis the older SmartBits platform. For example, with the port priority mode, packets can be scheduled according to a QoS policy thus simulating the complex traffic patterns often encountered in real-life networks. By using abstract objects such as host blocks and stream blocks instead of monolithic traffic tables, traffic patterns and volumes can be adjusted in real-time without having to stop and restart tests. With the command sequencer, testers now have the ability to automate tests through the use of a simple user interface. Finally, advanced analysis tools such as custom counters, template filters and real-time capture add an extra level of sophistication to the post test analysis, especially when a large number of test cases are involved. All of these will enable users to test complex network devices and systems more thoroughly and effectively. White Paper: Using Spirent TestCenter to Generate Real-World Traffic - 18 - January 2008 About the Author Mr. Angus Ma is the founder of AHM Technology Corporation, a consulting company based in Ottawa, Canada, that specializes in network testing and application performance monitoring. He can be reached at angusma@ahmtech.com.