Evaluating Performance Consistency for NFV- Based Services

Blog post image

Virtualization of network functions is an essential step for deployment of NFV implementations. Although most of the key network device vendors offer VNFs, providing consistent performance, reliability and resiliency using VNFs that is similar to the performance offered by traditional network devices is still a challenge.

Understanding Performance Consistency

Performance consistency can be thought of in terms of performance metrics such as throughput, latency, jitter and packet delay variation. It refers to the ability of the network function (NF) or Implementation Under Test (IUT) to deliver the same performance metrics over multiple test runs when subjected to same user workload. 

In traditional network device implementations, the challenge of deterministic performance and performance isolation is addressed by using ASICS, FPGAs and custom hardware that can deliver high performance in moving packets. The technologies involved in traditional network devices have evolved and enhanced over years, leveraging the advances in both hardware and software. Traditional network device implementations have dedicated hardware resources associated with network functions and minimal resource sharing is involved. Additionally, the resource allocation for network function is pretty much static and is not altered as more services are provisioned on the network node.

Delivering a similar type of performance consistency using VNFs in a virtualized environment is challenging because NFVI resources are shared across multiple VNFs and their allocation is dynamic. Performance delivered by a VNF may be impacted by number of factors such as underlying switching capacity of the virtual switch, NFVI node performance, NFVI resource allocation (whether VNF is provisioned on the same NUMA node, dedicated processor core assignment, dedicated memory assignment, external interrupt isolation) and the performance isolation capabilities offered by the virtualization layer.

Testing Performance Consistency


Figure 1: Virtual Switch Test Scenarios

In NFV environments, performance consistency comes under test when we are close to the upper limit of the virtual switch or NFV service chain. At that point, even low level of external interrupts, variation in user workload or resource sharing can impact consistency resulting in frame loss or variation in latency. 

Frame loss is generally observed only when a virtual switch or a NFV service chain is operating close to the upper limit of performance. For loss and delay intolerant services and applications, vendors may provide the ability to set threshold limits for throughput that are below the level when performance inconsistency kicks in. This also means VNF vendors providing separate Service Level Agreements (SLAs) for loss or delay intolerant services.

For loss tolerant services and applications, service providers (SPs) may be able to achieve higher performance limits by assuming a certain loss tolerance as long as the loss tolerance is within the SLA limits for the E2E service. This brings forward two sets of performance metrics, one for loss tolerant applications/services and other for loss intolerant services/applications.

Performance characterization for loss or delay tolerant services may not be a single number but instead a range that takes into account the SLA metrics of the target application/services. Although, benchmarking standards such as RFC 2544 are applicable for benchmarking virtual switches and associated service chains, the evaluation of performance consistency using RFC 2544 is still as manual a process as ever, prone to error, relying on the tester’s expertise. 

This creates the need for testing tools that help evaluate performance consistency in an NFV environment through automation. Since results based solely on test traffic may not provide enough information to understand the impact of NFVI resource sharing, an effective test tool must also provide results specific to the NFV environment itself. 

Spirent is leading the effort in providing not just a test tool for the NFV environment and SDN testing but a complete test solution that provides actionable results for NEMs and Service Providers, learn more here


comments powered by Disqus
× Spirent.com uses cookies to enhance and streamline your experience. By continuing to browse our site, you are agreeing to the use of cookies.