Crossbeam and Spirent Partner to Define Real-World Security Test Methodology for Mobile Network Operators
Jan 17, 2012
First Ever Tests – Validated by EANTC – Emulated More than a Million Subscribers Using Smartphones and Tablets, Provide Guidance for Building More Secure Mobile Networks
BOXBOROUGH, Mass. – Jan. 17, 2012 – Crossbeam and Spirent Communications today announced results of the industry’s first public network security tests that emulate more than a million simultaneous mobile subscribers using their smartphones and tablets. Conducted by EANTC, an internationally recognized independent test center, the tests evaluated the effectiveness of network security devices and how they perform under extremely demanding real-world conditions. Test results showcase what is necessary to credibly validate security performance, and provide reliable guidance for mobile operators as they seek to build network security infrastructure that can handle growing user demand for 4G/LTE networks.
Spirent, the leader in networks, devices and services testing; Crossbeam, a provider of high-performance network security platforms; and EANTC collaborated to define a set of real-world test scenarios that emulated the performance impact of network security being enabled in a mobile network environment.
“With so much misleading information about security product performance and the unpredictability of threats and traffic demands, there has been no clarity around what to test for and what methodology to use,” said Chet Gapinski, vice president, product development at Crossbeam. “Any single measurement, such as network throughput, is meaningless unless tested and measured in combination with other metrics under real-world conditions. We saw an important opportunity to work with Spirent and EANTC on a set of real-world tests to move the industry forward to provide more accurate, uniform guidance.”
In order to replicate the incredible traffic loads that carriers experience at the mobile Internet gateway (Gi/SGi), the market leading Spirent Avalanche was used to emulate more than 1 million users concurrently accessing the mobile data network with their connected devices. The traffic was sent through a single Crossbeam X-Series X80-S platform running Check Point software, where it underwent a series of firewall and IPS (intrusion prevention system) inspections, as well as network address translation (NAT).
Measuring subscriber happiness
The tests were designed to not only understand how a variety of security devices affect the performance on mobile network infrastructure – but also how these results translate into subscriber satisfaction and quality of experience. Due to the intense processing required to secure mobile data traffic, the user experience can be dramatically impacted through slow page load times, extended response times and connection instability.
“There is no silver bullet for measuring security performance,” said Jeff Schmitz, vice president of the Networks & Applications group at Spirent. “No one calls their provider to complain about connections per second. Instead, our goal was to create a test solution that truly emulates what happens on mobile operator networks, and provides realistic guidelines for building secure networks that can also support millions of connected devices with a high-quality user experience.”
The real-world traffic model was custom built to emulate a modern day mobile environment with more than 1 million active users. This included using a 95 percent majority of HTTP sessions representing a range of user profiles. In addition, email (SMTP and POP3) connections were emulated alongside Smart Phone OS over-the-air updates.
Using Spirent’s Avalanche layer 4-7 test solution, EANTC validated the Crossbeam X-Series at a groundbreaking 106 Gbit/s with stateful firewall, IPS and NAT enabled, with 1 million simultaneously active users, 4 million active TCP connections and 242,000 new connections per second.
In each test scenario – including firewall only, firewall and NAT, firewall and IPS, and NAT and IPS – every performance metric measured showed consistent service establishment performance at scale. Moreover, the user-experience measurements demonstrated the X-Series did not drop a single connection or request, and maintained a very low URL response time of 10 ms.
“We challenged the Crossbeam X-Series mobile security gateway with rigorous, objective and realistic performance tests,” said Carsten Rossenhövel, managing director, EANTC. “Mobile operators require dependable firewall performance and scalability figures for their fast growing 3G and LTE services. Crossbeam’s solution worked flawlessly and surpassed our expectations in the various mobile deployment scenarios we tested.”
To download a full report on the test results, visit http://www.crossbeam.com/why-crossbeam/performance