English
Benchmarking performance of network devices can be categorized in two dimensions – data plane and control plane. Traditionally, methods such as RFC 2544, RFC 6349, RFC 2889 have been used for data plane performance benchmarking and similarly, control plane benchmarking is conducted primarily in terms of protocol message response time, session setup time, route forwarding time, convergence measurements, session scale and route scale. While these methods are applicable for VNF benchmarking, they are not sufficient.
One of the key value propositions of the NFV paradigm is the ability to provision multiple VNFs from different vendors on the same compute node. In a shared infrastructure, multi-vendor environment, identifying performance bottlenecks becomes a tremendous challenge. Addressing this challenge is extremely tedious and expensive with traditional performance benchmarking methods. Additionally, the performance bottlenecks manifest itself based on the user workloads or services that are provisioned through the data path consisting of VNF service chains.
The SUT, in this case consists of compute node, VNFs provisioned on the compute node, hypervisor, NIC cards and virtual switch. Performance metrics that provide visibility into the virtual domain when the SUT is exposed to user workloads is extremely valuable. The ability to create user workload scenarios and correlate various types of user workloads with the NFVi utilization metrics goes a long way in not only characterizing the VNF performance but also in identifying the performance bottlenecks.
Currently, the test solutions available on the market either focus on emulating user workloads or focus on collecting NFVi resource utilization statistics with an incomplete understanding of which NFVi metrics impact VNF performance. A test solution that provides both and derives intelligence from the analysis and correlation of both network performance metrics (e.g. throughput, latency, jitter, session scale) and NFVi resource utilization statistics is extremely valuable for VNF performance benchmarking and characterizing VNF performance under various user workloads.
Let us look at a workflow of how a packet gets processed by a forwarding VNF.
In the workflow presented above there are multiple entities where performance bottlenecks in an NFV-based network may occur. NFVi metrics associated with virtual as well as physical entities can reveal important information about how effectively a VNF or virtual switch implementation utilizes the given NFVi resources or packet processing.
The ability to emulate realistic user workloads (both control plane and data plane) and the ability to understand factors impacting the performance of a VNF using NFVi analytics are essential attributes of effective VNF benchmarking solution.
Spirent is leading the industry in providing NFV test tools and test solutions that provide confidence to NFV vendors and Service Providers in moving from POCs and small-scale deployments to widespread NFV deployments.
To learn more about Spirent solutions for benchmarking and providing the associated analytics for both the control and data planes of NFVi visit: Spirent Testcenter
