Ensuring Performance & QoE with Network Impairment Emulation White Paper
Rev. A 0707
Inspired Innovation
White Paper
Network Applications
Ensuring Performance and QoE with
Network Impairment Emulation
July 2007
Spirent Communications, Inc.
1325 Borregas Avenue
Sunnyvale, CA 94089 USA
Email: sales-spirent@spirent.com
Web: http://www.spirent.com
Americas
T: +1 800.SPIRENT
+818 676.2683
Europe, Middle East, Africa
T: +33 1 6137.2250
Asia Pacific
T: +852 2511.3822
Copyright
© 2007 Spirent Communications, Inc. All Rights Reserved.
All of the company names and/or brand names and/or product names referred
to in this document, in particular, the name “Spirent” and its logo device, are
either registered trademarks or trademarks of Spirent plc and its subsidiaries,
pending registration in accordance with relevant national laws. All other registered
trademarks or trademarks are the property of their respective owners.
The information contained in this document is subject to change without notice
and does not represent a commitment on the part of Spirent Communications. The
information in this document is believed to be accurate and reliable; however,
Spirent Communications assumes no responsibility or liability for any errors or
inaccuracies that may appear in the document.
Spirent Communications White Paper
Network Applications
Ensuring Performance and QoE with Network Impairment Emulation
Contents
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Enterprise Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Network Application Challenges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Application Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Application Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Application Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Meeting the Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Predictive Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
In a Perfect World: Establishing the Baseline . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Real World Testing: The Reality Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Spirent Communications White Paper
Overview
Information management applications have progressed from an innovation to a
competitive advantage to essential infrastructure for enterprises. As applications
have matured, they have expanded from internal, LAN-based systems to distributed
systems deployed across the wide area network (WAN) and used by employees,
partners and customers with varying degrees of access.
Networked applications are pervasive in business and dependence on them is
increasing. An outage, or even poor performance, can cripple day-to-day business,
affecting revenues directly through lost sales and indirectly through decreased
productivity. Because of the proliferation of enterprise applications and the increased
complexity of systems and networks, it can be difficult to know how an application
will perform on any given network or to identify the actual cause of performance
issues when they arise. Because enterprise applications are rarely tested under WAN
conditions, expectations of performance are often at variance to actual performance.
Spirent’s Impairment Emulators are essential tools for effectively developing
and deploying enterprise applications. They provide the ability to accurately and
precisely emulate real-world network conditions prior to deployment, reducing
development schedules and support cost, avoiding live troubleshooting and
downtime, and consequently increasing revenue and competitive advantage.
Emulating your production network enables you to understand the behavior of an
application before it goes live, allowing you to determine if it will meet service
level objectives (SLO) and to correctly identify the factors affecting performance.
Network or application changes are based on data rather than guesswork or
conventional wisdom, assuring the performance, robustness and scalability your
organization needs in the initial rollout.
Emulating your
production network
enables you
to understand
the behavior of
an application
before it goes live
– allowing you to
determine if it will
meet service level
objectives (SLOs).
Network Applications—Ensuring Performance and QoE with Network Impairment Emulation
Overview
2Spirent Communications White Paper
Enterprise Applications
Distributed information systems have expanded beyond simple
accounting and inventory applications to address the wide
range of issues facing the enterprise, such as enterprise resource
planning (ERP), customer relation management (CRM), supply
chain management (SCM), business process management
(BPM), e-commerce, and other essential functions. Some
organizations use outside solutions, others develop their
own in-house solutions, and others integrate multiple best-of-
breed solutions.
The drivers for adopting and implementing enterprise
applications are ultimately financial – reducing costs,
increasing revenues and improving competitive advantage. By
consolidating functional silos in various locations, divisions or
companies into a centralized system, total cost of ownership for
IT systems can be reduced.
Architecture
Networked applications typically use a three-tier architecture
consisting of the data tier, the application tier and the client tier.
The data tier houses
the files and databases
containing the information
the system manages.
These are typically
SQL-based databases
or legacy data systems.
The application tier
communicates with the
data tier through an
application-programming
interface, typically
supporting CORBA,
DCOM, RMI, RPC
or a proprietary system.
To maintain business
continuity, the data is
protected through
off site backup or real-
time replication at a
remote location.
Drivers for
Enterprise
Applications
ERP
• Consistent
& accurate
organization-wide
financial reporting
• Consolidation of
customer order info
• Standardizing
manufacturing
processes
• Integrating
personnel data
across locations
CRM
Improving customer:
• Satisfaction
• Retention
• Profiling
• Targeting
SCM
• Removing
inefficiencies
• Streamlining
procurement
• Improving materials
quality
• Reducing costs
& inventory
BPM
• Increasing profit
margins through
performance
accountability
• Addressing
challenging market
conditions
• Corporate
governance issues
such as Sarbanes-
Oxley requirements
Data Tier
Three Tier Architecture
Application Tier
Client Tier
Transaction
Data Set Data Set
Response
Network Applications—Ensuring Performance and QoE with Network Impairment Emulation
Enterprise Applications
Spirent Communications White Paper
The application tier performs several functions and can be divided into sub-layers
that can be implemented on a single server or distributed among multiple servers.
Web and application servers reside in this tier, interfacing with the two other tiers
to accept a transaction request from the client, process the request, query the data
tier, and return the results to the client tier. Business logic resides in the middle tier
and includes implementation of work flows and business rules that distill policy,
such as pricing, logistics, or order processing, into algorithms. In keeping with
the distributed nature of the architecture, the servers implementing the application
tier may or may not be collocated with the data tier servers. In fact, servers
implementing various functions of the application tier can be located remotely from
each other.
The client tier is the user interface to the application. It can be a thin client (typically
a browser-based application using HTML or Java that doesn’t require installation on
the client computer), a fat client (an application that must be installed on the client
computer, typically implemented in C) or a hybrid. Many vendors offer both fat
and thin clients, with the momentum going toward thin. The client presents a screen
that is used to gather information from, and present results to, the user. To reduce
the amount of network traffic, the client often performs validation of input before
sending transactions to the application tier. While a client may be local to the facility
housing the application tier, most often clients are located remotely from the other
tiers and may be connected via a low-bandwidth dialup line or a high-bandwidth
broadband link.
Because of the
proliferation
of enterprise
applications and
the increased
complexity of
systems and
networks, it can be
difficult to know
how an application
will perform on
any given network
or to identify the
actual cause of
performance issues
when they arise.
Network Applications—Ensuring Performance and QoE with Network Impairment Emulation
Architecture
Spirent Communications White Paper
Network Application Challenges
Application Management
A single enterprise application is a complex system. Given the reality that most
organizations have multiple applications that may or may not be integrated with
each other, it’s no surprise that the complexity can quickly become unmanageable.
Most applications have some level of management available, but a single system
fault – a misconfigured server or a hitting a system limit – can generate multiple
alerts. An application expert must sort through the alarms in the context of system-
wide conditions to determine the underlying cause and propose a solution. Problems
of this nature occur dozens of times a day in enterprise data centers, making
troubleshooting and problem isolation a time-consuming and expensive task.
Application management software – whether built-in or third party – is designed to
assist IT managers by providing an early-warning system to identify issues before
they affect users. These systems are used to monitor performance metrics, automate
routine maintenance tasks and apply artificial intelligence to alarm situations to
assist in diagnosing root causes and identifying solutions.
Application Security
Enterprise applications manage vast amounts of sensitive information, including
proprietary designs, trade secrets, financial data, customer account data, and
personnel records with SSN and other data. Unauthorized access to such information
can have serious implications for an organization, including lost revenue, damaged
reputation and possible litigation.
Firewalls, encryption and user authentication are typically imposed at gateway points
but not inside the network. As enterprise applications go online, functions that were
once protected in back-office zones by physical access now have the potential to be
exposed to threats from the outside world.
Conversations between the application and data layers are usually assumed to be
secure and therefore applications often have unrestricted access to the database. But
the first level of security can be compromised through various methods, including
dynamic SQL commands, password cracking or configuration vulnerabilities such
as open SQL server ports. Once the perimeter is breached, the intruder can use the
application layer to gain high-level access to the database. For reliable security,
firewalls must be installed not only at the gateway, but also between each tier of
the architecture.
There is also risk from internal threats. A database user can be tricked into running
malicious code through a Trojan horse or a seemingly benign utility. Rogue users or
administrators can eavesdrop on the network to pick up passwords or data. For basic
protection, strict attention to security-focused configuration and regular updates and
patches for operating systems, server applications and database systems are essential.
In addition, encryption should be utilized between all layers.
A single enterprise
application
is a complex
system. Given the
reality that most
organizations
have multiple
applications that
may or may not
be integrated with
each other, it’s no
surprise that the
complexity can
quickly become
unmanageable.
Network Applications—Ensuring Performance and QoE with Network Impairment Emulation
Network Application Challenges
Spirent Communications White Paper
Application Performance
The distributed nature of enterprise applications can lead to performance problems.
In addition, the solution to one performance problem, such as compression to reduce
the number of bits transmitted, can create more performance problems.
• Congestion – The disparity between the bandwidth available on the LAN
versus the WAN can lead to congestion when moving large amounts of data
between locations, for example between the primary servers and off-site
mirror or back-up servers for disaster recovery and business continuity.
• Delay – Distance introduces delay problems, especially for connection-
oriented protocols like TCP, which require multiple exchanges to establish a
session and acknowledgments after each transmission.
• Processor-intensive functions – Compression is often used to reduce
congestion. Encryption is used to address security concerns. Both functions
place high demands on the server CPU that can affect the number of
transactions it can process.
Application performance problems can be addressed in two ways – by optimizing
the application for WAN communications and by deploying a WAN accelerator.
Application optimization is best done during development rather than as an
afterthought. Algorithms are designed to avoid chatty behavior, for example, by
compiling multiple requests into a single message.
As enterprise
applications go
online, functions
that were once
protected in back-
office zones by
physical access
now have the
potential to be
exposed to threats
from the outside
world.
Network Applications—Ensuring Performance and QoE with Network Impairment Emulation
Application Performance
Spirent Communications White Paper
WAN acceleration solutions reduce costs by increasing the effective throughput of
the existing network. Most solutions achieve a 10X to 40X improvement in response
time or transaction processing rate using a wide array of techniques.
• Compression – Typical compression techniques replace repeating bit
patterns with a short label before sending the data across the WAN. On the
receiving side the label is removed and the original pattern is inserted back
into the bit stream, reducing the number of bits transmitted by up to 70%
(Figure 1).
WAN
LAN A
WAN
Accelerator A
WAN
Accelerator B
Router A Router B
LAN B
No compression
With compression
A. No Compression
B. With compression
Figure 1. Compression
Too often,
application
performance isn’t
evaluated until
after deployment,
exposing the IT
department to
unwanted attention
when response
time doesn’t match
expectations.
Network Applications—Ensuring Performance and QoE with Network Impairment Emulation
Application Performance
7 Spirent Communications White Paper
• TCP Optimization – Transport Control Protocol (TCP) maintains the
end-to-end connection by requiring periodic acknowledgments from
the destination. A sliding window (the number of packets sent before
an acknowledgement is required) controls congestion (Figure 2). WAN
acceleration increases throughput by sending acknowledgments locally
rather than across the WAN (Figure 3).
WAN
LAN A LAN B
A
B
C
Transmit 1
Transmit 2
ACK 1
ACK 2
Retransmit 2
Retransmit 2
WAN
Accelerator A
WAN
Accelerator B
Router A Router B
Figure 2. Unoptimized TCP Connection
A. Normal transmt and acknowledgment results n ncreased wndow sze.
B. Lost packet, no acknowledgment causes retransmt and reduced wndow sze.
C. Excessve delay cause acknowledgment to arrve after tme-out, resultng n retransmt and reduced
wndow sze.
NOTE: Each transmt conssts of one or more IP packets, dependng on the TCP wndow sze. Each
acknowledgment s one IP packet.
WAN
LAN A LAN B
Transmit 1
ACK 1
Transmit 2
ACK 2
Transmit 3
ACK 3
Transmit 4
ACK 4
Transmit 5
ACK 5
Transmit 6
ACK 6
Transmit 1
ACK 1
Transmit 2
ACK 2
Transmit 3
ACK 3
Transmit 4
ACK 4
Transmit 5
ACK 5
Transmit 6
ACK 6
Transmit 1
Transmit 2
Transmit 3
Transmit 4
Transmit 5
Transmit 6
WAN
Accelerator A
WAN
Accelerator B
Router A Router B
Acknowledgments do not travel across the WAN and therefore do not experence long delays. More effcent
connecton protocols are used between the accelerator applances, whch are not pctured on the ladder dagram.
NOTE: Each transmt conssts of one or more IP packets, dependng on the TCP wndow sze. Each
acknowledgment s one IP packet.
Figure 3. Optimized TCP Connection
Testing provides
concrete data to
answer important
questions about
application
readiness and to
set performance
expectations,
which can then be
communicated to
management and
users.
Network Applications—Ensuring Performance and QoE with Network Impairment Emulation
Application Performance
Spirent Communications White Paper
• Caching and Mirroring – Client-side WAN accelerators store copies
of web pages and other data as it passes through so that subsequent
requests can be answered locally by the accelerator rather than requiring
retransmission from the remote server (Figure 4).
• Traffic Shaping through QoS – Delay-sensitive traffic, such as video or
voice, is prioritized and guaranteed delivery. Traditional data traffic,
such as e-mail or web page requests, are delivered if available bandwidth
permits or are dropped if the higher priority traffic has consumed the
available bandwidth.
• Off-loading CPU-Intensive
Tasks – By off-loading
CPU-intensive tasks, such
as compression, encryption
and XML processing, to
an appliance, organizations
cannot only improve
WAN performance but
also handle increased
traffic without the expense
of licensing additional
processors (Figure 5).
WAN
LAN A LAN B
Request 1
Response 1
Request 1
Response 1
Request 2
Response 2
Request 2
Response 2
WAN
Accelerator A
WAN
Accelerator B
Router A
A. Caching
B. No caching
Router B
Web
page
Web
page
Web
page
Web
page
A
NO CA
CHING
B
CA
CHING
Figure 4. Caching
WAN Accelerator
Server Server
E-mail Server
Web Server
Encryption
Compression
XML Processing
E-mail Server
Web Server
Encryption
Compression
XML Processing
Figure 5. Off-loading CPU Intensive Tasks
Too often,
application
performance isn’t
evaluated until
after deployment,
exposing the IT
department to
unwanted attention
when response
time doesn’t match
expectations.
Network Applications—Ensuring Performance and QoE with Network Impairment Emulation
Application Performance
Spirent Communications White Paper
• Forward Error Correction (FEC) – Some WAN acceleration solutions
use FEC to reduce the need for retransmissions on loss-prone networks.
FEC sends recovery information in-band with data for indexing to allow for
reconstruction of lost packets.
Too often, application performance isn’t evaluated until after deployment, exposing
the IT department to unwanted attention when response time doesn’t match
expectations. Troubleshooting becomes frantic. Meetings may become emotional
and politically charged. Users may accuse the IT department of using them as an
ad hoc QA department. Pressure to provide a quick fix may short circuit efforts to
identify root causes and effective solutions.
Meeting the Challenges
Performance testing and network emulation are key to successful enterprise
application deployment.
• Performance testing simulates the demands placed on the system by
generating realistic traffic/transaction loads and modeling realistic
user behavior
• Network emulation creates the expected environment by reproducing
realistic network conditions and behavior
Both are critical to predicting application performance before deployment.
Evaluating realistic traffic loads on a perfect network can create a false sense of
confidence that the application will perform to expectations after deployment.
Testing provides concrete data to answer important questions about application
readiness and to set performance expectations, which can then be communicated to
management and users.
Which part of the
system is the weak
link – web server,
application server,
database, and
network – and what
are the remedial
action items?
Questions Answers: Examples
What are the applcaton performance thresholds n
terms of network metrcs (delay, packet loss), number
of transactons or number of users?
Response tme falls off drastcally when packet loss exceeds %.
Whch part of the system s the weak lnk – web server,
applcaton server, database, and network – and what
are the remedal acton tems?
Delay above 200 ms creates TCP tme outs and retres, resultng n
response tmes beyond SLO. A WAN accelerator can elmnate TCP
connecton ssues for remote offces, but doesn’t address ssues
for moble users.
Can the applcaton meet SLO under all expected
profles, such as remote offce, home offce or busness
travel access?
Users can expect second response tmes when connectng
va DSL.
Does the new verson solve performance ssues or
ntroduce lmtatons not present n the earler verson?
Regresson testng aganst our reference network emulaton
profles demonstrates a 2% mprovement n response tme for
remote users.
Wll the proposed fx actually mprove performance?
The developers argue that the network, not the applcaton, s the
source of performance ssues. Bandwdth lmtaton testng shows
no sgnfcant performance enhancement when WAN bandwdth
s ncreased.
Network Applications—Ensuring Performance and QoE with Network Impairment Emulation
Meeting the Challenges
0Spirent Communications White Paper
Predictive Performance Evaluation
In a Perfect World: Establishing the Baseline
As the first step in producing a realistic performance prediction – the baseline
– establishes how the application performs in a perfect world, or at least on a perfect
network.
The QA team studies the organization to assess periodic and event-driven usage
and behavior changes, such as time-of-day variations, end-of-month/quarter/year
demands and factors that may be unique to the organization or industry. The results
form the basis of a set of traffic-load and user-behavior profiles that reflect the range
of expected usage patterns. These profiles should include the login storms that occur
in the morning and after-lunch, typical usage with full attendance, and peak usage
during crunch times, such as end-of-period reconciliation and reporting. The profiles
will be used during baseline testing and network emulation testing.
The application is first tested with the load/behavior profiles under optimum network
conditions. The infrastructure of the test lab provides an environment with no packet
loss, no packet reorder and minimal delay. The metrics reported from each profile
establish the application performance baseline, which is compared to SLO. An
application that fails or barely passes SLO in an optimum network will not have
acceptable performance across the WAN.
Next, failure thresholds are established. Threshold testing doesn’t use typical load/
behavior profiles. Instead, traffic loads are ramped up in steps and failure points
(the level at which the performance metrics violate SLO) due to the limits of the
application are identified. Threshold testing helps predict scalability limits and
allows management to plan for growth, identifying points where investment in
infrastructure will be required.
Profiles should
include the login
storms that occur
in the morning and
after-lunch, typical
usage with full
attendance, and
peak usage during
crunch times.
Test Type
Baselne Load Profles
Load threshold
Network Emulaton Load profles
Load threshold
Imparment threshold
Network Applications—Ensuring Performance and QoE with Network Impairment Emulation
Predictive Performance Evaluation
Spirent Communications White Paper
Real World Testing: The Reality Check
Once the application achieves acceptable performance (baseline) and margins
(threshold), the next phase of testing begins. Each of the tests performed during
baselining are performed again under more realistic conditions.
The network team provides a set of network profiles that reflect the characteristics of
each type of user.
• An on-site profile will have LAN speeds and minimal impairment
• A remote office profile will have WAN speeds and more impairment
• A home office profile might have broadband access speeds and delays for
VPN security
• A mobile user may have a variety of connection options, including dialup,
either directly into the company network or to a provider and then via VPN,
or broadband with VPN
The application is tested using each load profile with each network profile. The
metrics reported from each test are compared to SLO to verify that the application
can still deliver acceptable performance under the expected network conditions.
Violations of SLO are subjected to troubleshooting to determine the root cause and
remedial measures.
During network emulation testing, two types of failure thresholds are established.
As during baselining, traffic load thresholds are measured, but during network
emulation testing they are established for each network profile. The results are
compared to the baseline thresholds to right-size scalability limits for future growth.
In addition, impairment thresholds are identified. Network delay and impairment
settings are ramped up in steps, and failure points due to network conditions are
identified. Network impairment threshold testing helps identify the service level
agreement terms required from WAN vendors to assure acceptable application
performance for remote users.
Each step of application readiness testing provides valuable information for design
review, application optimization, strategic and tactical planning, budgeting, level
setting of expectations, reporting to management, and troubleshooting.
Each step of application
readiness testing
provides valuable
information for:
• Design review
• Application
optimization
• Strategic & tactical
planning
• Budgeting
• Level-setting of
expectations
• Reporting to
management
• Troubleshooting
Network Applications—Ensuring Performance and QoE with Network Impairment Emulation
Real World Testing: The Reality Check
2Spirent Communications White Paper
Conclusion
Enterprise applications are essential elements of the business infrastructure. Poor
application performance can cripple day-to-day business and affect productivity.
The complexity of enterprise applications and networks makes it difficult to predict
application performance and identify the root cause of performance issues.
The key to a successful application rollout is robust testing under realistic traffic and
network conditions. The two-tier testing model of establishing baseline and network
emulation metrics and determining load and impairment thresholds empowers you
to compile the data you need to make informed decisions and communicate realistic
expectations vertically and horizontally through your organization.
Spirent’s Impairment Emulators are essential tools for effectively developing and
deploying enterprise applications. They provide the ability to:
• Accurately and precisely emulate real-world network conditions prior
to deployment
• Efficiently test different network user profiles
• Reduce development schedules and support cost
• Avoid live troubleshooting and downtime
• Increase revenue and competitive advantage
Network impairment testing can mean the difference between a smooth deployment
and a crisis. Make sure you have the tools required for a successful rollout.
Network emulation
testing can mean
the difference
between a smooth
deployment and
a crisis.
Network Applications—Ensuring Performance and QoE with Network Impairment Emulation
Conclusion
Inspired Innovation
