Signals Research Group’s New LTE Smartphone Benchmark Study Finds Wide Variations in Data Performance

Sunnyvale, Calif.– (Thursday, January 30 2014) — Spirent Communications, the leader in testing networks, services and devices, announced today  it has  collaborated with Signals Research Group (SRG) and ETS-Lindgren on the first independent Over-the-Air (OTA) benchmark study of commercial LTE smartphone performance. Results of SRG’s study illustrate the unique ability of the OTA test methodology to detect clear data performance differences between commercial LTE device implementations, as well as confirming that poorly-performing smartphones make inefficient use of network resources and can significantly degrade the mobile user experience.

“Once a great-performing chipset is embedded into a mobile device, there are innumerable factors that could result in the smartphone performing poorly with respect to cellular connectivity, including implementation of MIMO antennas,” commented Michael Thelander, CEO of Signals Research Group. “For this reason we decided to conduct this latest round of benchmark testing over the air, as this is perhaps the best methodology available today to fully characterize device performance.”

Key conclusions of the study:

• A poor-performing smartphone can dramatically degrade the user experience, make inefficient use of valuable network resources and increase subscriber churn.

• Data throughput was generally substantially higher with MIMO than without, but the benefits were far less meaningful with less favorable network conditions.

• Although far from conclusive, the study found that a protective cover can have a modest impact on the performance of a smartphone, in particular when using MIMO.

“Our study found substantial OTA cellular performance differences in the smartphones tested,” added Thelander. “However the results show that one device –  the Moto X from Motorola Mobility –  came out on top by a country mile.”

The study included OTA tests for LTE data throughput performance with MIMO for some of the industry’s most popular mobile devices, all of which used chipsets from a single manufacturer. Past SRG studies utilized traditional lab-based conducted testing, which provide valuable information on device and chipset performance, but do not deliver the complete cellular performance picture that can be achieved with OTA testing. OTA testing is carried out in a controlled radiated environment, in this case a cost-effective anechoic chamber, exposing the device under test to radiated signals from probes within the chamber while isolating the device from external elements that could influence the cellular performance results.

“OTA test methodology is the only way to thoroughly assess the real-world data performance of LTE devices,” said Bryan Sayler, Senior Vice President ETS Lindgren. “Performance is highly dependent on device design, especially with complex antenna schemes such as MIMO. As LTE continues to roll out we believe that the industry will place much greater reliance on testing mobile device performance over the air, utilizing an approach similar to the one used in this important SRG study.”

“We were thrilled to partner with SRG and ETS to showcase the value-add of OTA test methodology when it comes to accurately evaluating and ultimately improving the overall user experience,” said Nigel Wright, Vice President Spirent Communications. “In benchmarking commercial devices that all used chipsets from a single vendor, we exposed the reality that use of a top-performing chipset does not necessarily guarantee a top-performing mobile device. The industry has a significant and complex challenge to address when validating LTE device performance to ensure a successful commercial rollout.”

This study utilized Spirent’s CS8 LTE Network Emulator, VR5 channel emulator, Spirent DatumLab server and ETS Lindgren’s new AMS-8055 MIMO Single Cluster Environment Simulator as a cost-effective approach to conducting the OTA tests. All devices evaluated were commercially-available and procured directly from the mobile operator’s store or website.  None of the handset manufacturers included in the study had any prior knowledge of the study or its results until after the report was published.

For a preview of the report in Signals Ahead, “Chips and Salsa XVII: When Iconic Meets Anechoic” please visit the consultancy’s website at www.signalsresearch.com.