Testing Triple PlayServices Application Note
P/N 71-002670 Rev. A 0607
Inspired Innovation
Application Note
Spirent Network and
Impairment Emulators
Testing Triple Play Services
Streaming Video• VoD • Multicasting • RTP • MGCP • MEGACO • H.323
June 2007
Spirent Communications, Inc.
1325 Borregas Avenue
Sunnyvale, CA 94089 USA
Email: sales-spirent@spirent.com
Web: http://www.spirent.com
Americas
T: +1 800.SPIRENT
+818 676.2683
Europe, Middle East, Africa
T: +33 1 6137.2250
Asia Pacific
T: +852 2511.3822
Copyright
© 2007 Spirent Communications, Inc. All Rights Reserved.
All of the company names and/or brand names and/or product names referred
to in this document, in particular, the name “Spirent” and its logo device, are
either registered trademarks or trademarks of Spirent plc and its subsidiaries,
pending registration in accordance with relevant national laws. All other registered
trademarks or trademarks are the property of their respective owners.
The information contained in this document is subject to change without notice
and does not represent a commitment on the part of Spirent Communications. The
information in this document is believed to be accurate and reliable; however,
Spirent Communications assumes no responsibility or liability for any errors or
inaccuracies that may appear in the document.
Spirent Communications White Paper
Spirent Network and
Impairment Emulators
Testing Triple Play Services
Contents
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
The Technology: Video . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
The Technology: Voice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Testing Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
In a Perfect World: The Performance Baseline . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Real-World Testing: The Reality Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Before and after . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Quality of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Voice Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Video Quality. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Buffer Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
PLC Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Channel Changing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Performance Thresholds. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1 Spirent Communications White Paper
Introduction
The delivery of voice, video and data over an Internet Protocol (IP) network
known as Triple Play provides service providers with new avenues for growth. The
technology enables more intelligent services such as simultaneous viewing/recording
of multiple programs, targeted advertising, on-screen caller ID and content control.
In addition, providers have greater flexibility in bundling and marketing a range of
services.
However, new opportunities bring new challenges. Delivering voice and video over
IP requires time-sensitive content to be sent over a network that may introduce
packet delay, loss and reordering with limited bandwidth. A successful Triple Play
solution must provide Quality of Experience (QoE) by compensating for negative
conditions that occur in IP networks. Spirent Impairment Emulators allow test
engineers to introduce those conditions in a controlled and repeatable fashion to
predict QoE in actual networks.
The Technology: Video
IP Video typically uses MPEG-2 encapsulated in Real Time Streaming Protocol
(RTSP) and IP. Video is a set of individual frames played at a rate sufficient to be
perceived as smooth motion, typically 30 frames per second. However, transmitting
30 full frames per second requires more bandwidth than is practical on IP networks.
The Moving Pictures Experts Group (MPEG) developed methods of reducing the
required bandwidth by reducing the amount of information required to display the
next frame in a sequence.
MPEG compression divides video into a set of frames called a group of pictures
(GOP). A GOP comprises three types of frames: I-frame (intra-frame), P-frame
(predictive frame) and B-frame (bidirectional frame). An I-frame contains all the
information required to display a frame.
Spirent Network and Impairment Emulators—Testing Triple Play Services
Introduction
2Spirent Communications White Paper
A P-frame contains only changes from the preceding I-frame – information on the
parts of the picture in motion, but no information on the parts of the picture that
are static. The information contained in a P-frame is not sufficient to display a full
image. It must be preceded by an I-frame.
A B-frame is similar to a P-frame, but contains information that has changed from
the preceding frame or will change in the following frame. In a GOP, I-frames are
followed by P-frames and B-frames. The more I-frames, the better the video quality.
I-frames are two or three times larger.
A typical GOP will contain seven to ten frames in the pattern IBBPBBPBBP,
although the order may be adjusted before transmission. A GOP is encapsulated into
one IP packet.
I1 B1 B2 P1 B3 B5 B6 P3 B7 B8 I2B4 P2
G.O.P.
MPEG Frames
Transmission Order
I1 B1
Bidirectional coding
B2 P1 B3 B5 B6 P3 B7 B8 I2B4 P2
Spirent Network and Impairment Emulators—Testing Triple Play Services
The Technology: Video
Spirent Communications White Paper
The Technology: Voice
In addition to the Internet Protocol (IP), other protocols make VoIP possible. They
can be divided into call setup and transport of the call itself.
Transport Control Protocol (TCP) is the reliable-transport partner in the TCP/IP
protocol suite. As a result, it is used to encapsulate call-signaling protocols to verify
the call is established. However, TCP has too much overhead to efficiently transport
the actual call. User Datagram Protocol (UDP), a connectionless transport protocol,
is used to carry the voice packets.
During call setup, a TCP connection between the two endpoints is established. This
can involve the use of proxy, registration and redirect servers to locate the IP address
of the called party. Once the TCP session is in place, the ring (called party) and
ring back (calling party) tones are triggered and the parameters governing the call
are negotiated. These can include master/slave settings for controlling the session,
choice of codec (coder-decoder) and compression scheme, the type of call (voice/
video conference) and security settings such as encryption. Call setup/signaling
protocols include H.225/Q.913, H.245, H.323, SIP, SDP, MGCP and MEGACO.
H.323
terminal
H.323
terminal
H.323
terminal
H.323
terminal
Branch Office
H.323 Gatekeeper
and Gateway
Site One
H.323 Gatekeeper
and Gateway
Site Two
H.323 Gatekeeper
and Gateway
Telecommuter
H.323 terminal via
high-speed connection
H.323
terminal
H.323
terminal
ISP Network
To PSTN
Voice
To other
sites
High-Speed
Data Network
Spirent Network and Impairment Emulators—Testing Triple Play Services
The Technology: Voice
Spirent Communications White Paper
Once the call is established, the conversation is encoded or digitized (converted from
analog signals to bits), compressed and split into groups of bytes to be encapsulated
into payloads for transport across the network. It may also be encrypted. Real-time
Transport Protocol (RTP) maintains packet sequence, since UDP doesn’t have the
sequencing support provided by TCP. Real-time Transport Control Protocol (RTCP)
is the companion protocol that provides out-of-band control information for an RTP
flow. RTCP can provide feedback on Quality of Service (QoS) for the flow to a
VoIP application, which can adjust differentiated services (DiffServ) parameters for
the flow.
Testing Considerations
Triple Play is a complex application requiring the seamless operation of many
functions. The transport must deliver the required bandwidth at appropriate service
levels. Voice conversations must be sampled, coded and decoded, compressed and
decompressed, packetized and depacketized, encrypted and decrypted. Signaling
is required to establish voice connections and negotiate settings. More signaling is
required to select video channels and Video on Demand programming. The system
must be scalable to accommodate a sustainable customer base and the associated
stress of channel zapping during peak times. Most importantly, the system must be
able to deliver high voice and video quality over the deployed infrastructure. And,
all of this must be delivered in the presence of data traffic.
In a Perfect World: The Performance Baseline
The first step in testing a Triple Play solution is to evaluate it in a test lab under
optimum conditions. The infrastructure of the test lab should provide an environment
with no packet loss, no packet reorder and minimal delay. As all aspects of the
solution are tested, such as signaling capacity, voice capacity, voice quality (PESQ/
PSQM MOS scores), channel zapping, video quality, QoS schemes and scalability,
the maximum performance limits of the system are determined. These limits
establish the baseline to be used for comparison against the results of tests in a more
realistic environment.
Real-World Testing: The Reality Check
Triple Play services are not delivered on an ideal network, but with other traffic
through multiple devices and across infrastructure that spans significant distances.
The result is an environment that presents some level of delay, impairment and
bandwidth limitation, the severity of which depends on how well the network is
engineered.
To deliver Triple Play with confidence, test and debugging iterations are performed
under conditions in which the solution will actually be deployed, and under worse
conditions to account for equipment failure and disasters. Rather than test on a live
network – which is expensive, impractical and not repeatable real-world testing is
achieved by using network emulation in the test lab.
Spirent Network and Impairment Emulators—Testing Triple Play Services
Testing Considerations
Spirent Communications White Paper
The tests performed during baselining are performed again with a Spirent Network
and Impairment Emulator creating impairments such as packet loss, delay, bit errors,
sequence errors or duplication in a controlled fashion. The emulator can also model
the effects of high traffic load by constraining available bandwidth. Actual values
used are determined by various methods.
Determnng Network Emulaton Parameters
Educated Guess
Trial and Error
Values known to affect the service can be tweaked to determine QoE
thresholds. For example, MDI and V-Factor use jitter, sequence errors
and packet loss, among other metrics, to measure video quality. Values
for these impairments can be ramped up, both individually and jointly, to
determine the breaking point for video quality scores. Results can be used to
characterize the limits of a device or service.
Pros: Granular control over specific impairment metrics, goal-seeking
of performance thresholds.
Cons: Can be time-consuming if not automated; combinations tested
may not reflect real-world conditions.
Network
Management System
or Protocol Analyzer
A network management system or protocol analyzer can determine the
performance of the live network on which the solution will be deployed.
Measured values for packet loss, delay, duplication and sequence errors can
be used to configure the emulator during testing.
Pros: Test conditions reflect the actual conditions measured on the
production network.
Cons: Conditions in IP networks are not static. They vary over time
as events such as traffic congestion, router flaps and topology
changes occur. Testing with a single value averaged over time
will not produce an accurate reflection of actual performance.
Realistic testing requires impairments that change dynamically
over time. Taking thousands of measurements over short
intervals and automating the changes in configuration of the
network emulator can accomplish this, but it can be complex
and time consuming.
Standards-Based
Model
A standard network model for IP network impairment can be used. TIA-921
and ITU-T G.1050 describe a Network Model for Evaluating Multimedia
Transmission Performance over Internet Protocol based on actual network
data provided by anonymous IP service providers and IP network equipment
manufacturers. An algorithm uses typical LAN, access and core network
characteristics as input and produces as output dynamic values for end-to-
end packet delay, loss and reorder that vary over time in the same manner as
conditions in deployed networks.
Pros: The model more accurately reflects the way actual IP networks
behave compared with other methods of testing.
Cons: Since the impairments change dynamically, in the case of failure
of a test case, the impairment that caused the failure may not be
immediately clear and further troubleshooting may be required.
Spirent Network and Impairment Emulators—Testing Triple Play Services
Testing Considerations
6Spirent Communications White Paper
Before and After
Features susceptible to specific impairments are baselined and tested in the presence
of the impairments known to affect them. As issues are revealed, troubleshooting
uncovers root causes and facilitates the debugging of applications or tuning of
network parameters. This iterative process assures robust solutions and minimizes
support costs.
Quality of Service
Real-time traffic is sensitive to packet loss. While packet-loss concealment (PLC)
algorithms help to mask the effect of missing packets, there is a threshold beyond
which PLC is ineffective. For video, the dropping of a single
MPEG I-frame is noticeable. QoS is employed to guarantee
delivery of voice and video with minimum latency. Different
schemes may be used depending on implementation. Queues
for real-time traffic must deliver packets with minimum loss,
even in the presence of other, lower priority traffic. Bandwidth
oversubscription should be tested to verify that lower priority
traffic is dropped to accommodate voice and video.
Impairment profiles can be configured based on where the
QoS is implemented, such as in the experimental bits in the
MPLS header (core network traffic engineering), the DSCP/ToS bits in the IP header
(aggregation-level QoS) or the 802.1p-bits in the VLAN tag (DSLAM or PON QoS).
Voice Quality
Voice quality can be affected by network jitter, sequence errors, jitter discards
(frames discarded due to arriving too late) and packet loss. Implementations
are tested against multiple loss profiles, from
periodic and random loss rates to burst loss events
associated with traffic congestion, route failure and
route flapping. Acceptable quality thresholds are
established for each loss profile.
Spirent Network and Impairment Emulators—Testing Triple Play Services
Before and After
7 Spirent Communications White Paper
Video Quality
QoE measures of experience for IP video can be categorized into those that measure
network statistics and those that measure the content itself. Media Delivery Index
(MDI), specified in IETF RFC 4445, predicts the quality of packet video by
measuring network statistics such as packet jitter, loss and sequence errors. V-Factor
assesses the quality of packet video by inspecting
the content of the MPEG frames themselves. Both
scores will be affected by emulation of network jitter,
sequence errors, jitter discards (frames discarded
due to arriving too late) and packet loss. However,
MDI will not detect corruption of video content if
that corruption doesn’t result in a discarded packet.
An impairment emulator that modifies payloads with
checksum correction or drop specific frames (such as
I-Frames) will test this scenario. Results will mirror
those of a V-Factor test.
Buffer Size
Buffers are targeted specifically to addressing issues of mis-ordered packets and
jitter. Quality or performance scores from baseline tests should be compared to
results as re-order and jitter are applied in a controlled fashion. As breaking points
are established, adjustments to buffering can be tested to known values to discover
optimum buffer design and configuration.
PLC Algorithms
PLC is targeted at packet loss. Implementations are tested against multiple loss
profiles, from periodic and random loss rates to burst loss events associated with
traffic congestion, route failure and route flapping. Performance thresholds are
established for each loss profile.
Channel Changing
IP Video uses IGMP and multicast to efficiently transmit channels to multiple
destinations. If Join/Leave requests are corrupted or dropped, it will affect channel
zapping as requests are re-transmitted. The Anue GEM Network Emulator can be
used to target IGMP requests to measure the effect of dropped requests.
Spirent Network and Impairment Emulators—Testing Triple Play Services
Before and After
Spirent Communications White Paper
Performance Thresholds
When baselining performance, packet loss and sequence errors caused by the system
under test may increase as thresholds are reached. When testing with impairments,
the amount of impairment injected should be compared to the impairment measured
by the analyzer to compare performance thresholds to the baseline thresholds.
In addition, performance thresholds under realistic conditions determine service
level parameters required for a sustainable implementation. Testing with network
emulation offers a major advantage. It right sizes the engineering of a production
network by avoiding needless and expensive over-engineering and eliminating the
revenue-threatening consequences of under-engineering the network.
Conclusion
A successful Triple Play solution requires testing under real-world conditions to
guarantee that it can provide viewer QoE by compensating for conditions in IP
networks. Spirent Impairment Emulators enable engineers to predict application
performance thereby reducing risk, accelerating time to market, and allowing for a
smoother deployment with fewer problems and greater confidence.
Spirent Network and Impairment Emulators—Testing Triple Play Services
Conclusion
Inspired Innovation
