Testing Data Center Fabrics for High Performance at Massive Scale
How do you test a data center switching fabric that scales exponentially beyond legacy platforms?
By Jurrie van den Breekel
The implications go beyond the QA department. Of course, system testing is required as part of the product lifecycle and a large-scale system requires large-scale testing. But such a system poses an equally daunting challenge to marketing and sales—demonstrating to customers that high performance and high scalability can be achieved in their environment.
Independent test lab Network Test recently used Spirent’s data center testing solution to demonstrate that Juniper Networks’ QFabric™ can deliver unprecedented performance at scale.
Legacy switch fabrics scale to hundreds of ports. The QFabric solution scales to thousands. A fully loaded QFabric system supports 6,144 ports configurable as a single device in an any-to-any port topology with low latency and the appearance of unlimited bandwidth.
In addition to high speed, high-port density, and the ability to measure latency with precision, a proper test of such a fabric requires a full-mesh network topology traffic pattern. It is the most demanding possible configuration both for the test platform and the device under test, but it more accurately reflects the ultimate limits of system performance, an important consideration in a highly-dynamic environment where the demands of the next killer app are not known.
The test bed for this industry-first test comprised 1,536 x 10 GbE ports connected to the same number of Spirent TestCenter test ports and 128 redundant 40 Gbps fabric uplinks, a test configuration four to five times larger than previous high-port density Ethernet switch tests. Juniper QFabric System components took two 42U racks. Spirent TestCenter chassis loaded with 32-port HyperMetrics DX modules filled the other two racks.
The test methodology included standards-based benchmarking tests for unicast throughput, unicast latency, and multicast performance. Because of the full-mesh topology, Spirent TestCenter generated and analyzed throughput and latency over 2.3 million streams in real time and over 15 Tbps of traffic at line rate. Network Test used Spirent TestCenter’s analysis of Juniper Network's QFabric to validate that:
- Throughput for Layer 2 traffic was virtually identical in store-and-forward and cut-through modes, with rates approaching the maximum channel capacity for most frame sizes
- QFabric forwarding delay is low and consistent across all tests (less than 5 microseconds for all frames sizes up to 512 bytes) when offered loads below the throughput rate
- Average latency for Layer 3 traffic is 10 microseconds or less for most frame sizes tested
- Multicast throughput was close to line rate in all tests, regardless of frame length, with the system moving traffic at speeds of up to 15.3 terabits per second
- Multicast average latency was low and consistent in all tests, never exceeding 4 microseconds
- The QFabric System successfully interoperated with Cisco Nexus 7010 and Cisco Catalyst 6506-E switch/routers when using common data center protocols such as link aggregation, OSPF equal cost multipath, and BGP
For more information on the individual tests methodologies, configurations, and detailed test results, see the test report from Network Test:
Additional resources of interest: