Secure Virtual Data Centre Testing Report
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 1
Secure Virtual Data
Centre Testing
- Featuring HP TippingPoint’s
Secure Virtualization Framework
A Broadband-Testing Report
By Steve Broadhead, Founder & Director, BB-T
Secure Virtual Data Centre Test
2 © Broadband-Testing 1995-2010
First published September 2010 (V1.0)
Published by Broadband-Testing
A division of Connexio-Informatica 2007, Andorra La Vella
Tel : +376 633010
E-mail : info@broadband-testing.co.uk
Internet : HTTP://www.broadband-testing.co.uk
2010 Broadband-Testing
All rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of the
authors.
Please note that access to or use of this Report is conditioned on the following:
1. The information in this Report is subject to change by Broadband-Testing without notice.
2. The information in this Report, at publication date, is believed by Broadband-Testing to be accurate and reliable, but is not guaranteed. All use of and reliance on
this Report are at your sole risk. Broadband-Testing is not liable or responsible for any damages, losses or expenses arising from any error or omission in this
Report.
3. NO WARRANTIES, EXPRESS OR IMPLIED ARE GIVEN BY Broadband-Testing. ALL IMPLIED WARRANTIES, INCLUDING IMPLIED WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT ARE DISCLAIMED AND EXCLUDED BY Broadband-Testing. IN
NO EVENT SHALL Broadband-Testing BE LIABLE FOR ANY CONSEQUENTIAL, INCIDENTAL OR INDIRECT DAMAGES, OR FOR ANY LOSS OF PROFIT,
REVENUE, DATA, COMPUTER PROGRAMS, OR OTHER ASSETS, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
4. This Report does not constitute an endorsement, recommendation or guarantee of any of the products (hardware or software) tested or the hardware and
software used in testing the products. The testing does not guarantee that there are no errors or defects in the products, or that the products will meet your
expectations, requirements, needs or specifications, or that they will operate without interruption.
5. This Report does not imply any endorsement, sponsorship, affiliation or verification by or with any companies mentioned in this report.
6. All trademarks, service marks, and trade names used in this Report are the trademarks, service marks, and trade names of their respective owners, and no
endorsement of, sponsorship of, affiliation with, or involvement in, any of the testing, this Report or Broadband-Testing is implied, nor should it be inferred.
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 3
TABLE OF CONTENTS
TABLE OF CONTENTS ..................................................................................... 3
BROADBAND-TESTING .................................................................................. 5
EXECUTIVE SUMMARY ................................................................................... 6
INTRODUCTION: THE ISSUES OF SECURING A VIRTUAL DATA CENTRE ........... 8
DEFINING OUR VIRTUAL DATA CENTRE TESTING: THE BIGGER PICTURE ...... 11
HP TIPPINGPOINT SECURE VIRTUALIZATION FRAMEWORK ......................... 13
SVF Overview ...................................................................................... 13
HP TippingPoint 5100N Intrusion Prevention System (IPS) ................ 16
PUT TO THE TEST ........................................................................................ 17
Test Bed Overview ............................................................................... 17
Our Test Findings ................................................................................. 19
External To External: Basic Throughput ......................................... 19
Performance Testing From External (Physical) To Internal (Virtual) Environments
.............................................................................................. 19
Performance In The Virtual Environment ....................................... 20
External To Internal: Scalability/Application Availability .................... 23
Scalability/Application Availability In The Virtual Environment ........... 23
Security: Attacking The Virtual Environment .................................. 24
External And Internal Combined Traffic/Attacks .............................. 27
SUMMARY & CONCLUSIONS ......................................................................... 28
APPENDIX 1: THE VIRTUAL DATA CENTRE COMPONENTS WE CAN TEST ........ 29
APPENDIX 2: SPIRENT AVALANCHE VIRTUAL ............................................... 31
Figure 1 – Virtual Data centre OS - VMware...........................................................................................................................8
Figure 2 – HP TippingPoint SVF in Data Centre Deployment .....................................................................................................9
Figure 3 – A Solar Powered Data Centre In The US ............................................................................................................... 11
Figure 4 – A Proposed Virtual Data Centre Topology ............................................................................................................. 12
Figure 5 – HP TippingPoint SVF .......................................................................................................................................... 13
Figure 6 – HP TippingPoint vController Environment: As Tested ............................................................................................. 14
Figure 7 – HP TippingPoint 5100N IPS................................................................................................................................. 16
Figure 8 – Our Test Bed (Overview) ................................................................................................................................... 17
Figure 9 – Spirent Avalanche 3100 ..................................................................................................................................... 17
Figure 10 – External Tests: Throughput Levels of 5100N ....................................................................................................... 19
Figure 11 – External To Internal Tests: Peak Throughput Levels............................................................................................. 20
Figure 12 – Internal Tests: Maximum Throughput ................................................................................................................ 21
Secure Virtual Data Centre Test
4 © Broadband-Testing 1995-2010
Figure 13 – Internal Tests: Throughput With Redirector Disabled .......................................................................................... 21
Figure 14 – Internal Tests: Throughput – vController Redirector Enabled ................................................................................ 22
Figure 15 – Internal Tests: Scalability – Server Response Times ........................................................................................... 23
Figure 16 – Internal Tests: Security – Code Red Test ........................................................................................................... 24
Figure 17 – Internal Tests: Security – Multi-Threat Attack .................................................................................................... 25
Figure 18 – Internal Tests: Security – Multi-Threat Attack Plus Firewall Filter .......................................................................... 26
Figure 19 – External-Internal Test: Security – Multi-Threat Attack with 10Gbps Traffic ............................................................. 27
Figure 20 – Spirent Avalanche Virtual ................................................................................................................................ 31
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 5
BROADBAND-TESTING
Broadband-Testing is Europe’s foremost independent network testing facility and consultancy
organisation for broadband and network infrastructure products.
Based in Andorra, Broadband-Testing provides extensive test demo facilities. From this base,
Broadband-Testing provides a range of specialist IT, networking and development services to
vendors and end-user organisations throughout Europe, SEAP and the United States.
Broadband-Testing is an associate of the following:
Limbo Creatives (bespoke software development)
Broadband-Testing Laboratories are available to vendors and end-users for fully independent
testing of networking, communications and security hardware and software.
Broadband-Testing Laboratories operates an Approval scheme which enables products to be
short-listed for purchase by end-users, based on their successful approval.
Output from the labs, including detailed research reports, articles and white papers on the latest
network-related technologies, are made available free of charge on our web site at
HTTP://www.broadband-testing.co.uk
Broadband-Testing Consultancy Services offers a range of network consultancy services
including network design, strategy planning, Internet connectivity and product development
assistance.
Secure Virtual Data Centre Test
6 © Broadband-Testing 1995-2010
EXECUTIVE SUMMARY
head2right The virtual/cloud world is with us – like it or not. Vendors are promoting the
solution to such an extent that deployment is inevitable as the hand of the user is
being forced.
head2right There is a real fear of uptake in virtualization, based around what is perceived as
a potential security problem, none more so than at the heart of a virtual data
centre.
head2right Spirent has already proven that it can test in the cloud – see Cloud Computing
Report carried out in conjunction with EANTC - http://www.eantc.de/public-
reports.html - so here we focus on securing the virtual data centre.
head2right The classic methodologies used for testing over the years are only capable of
telling a small part of the story here. Testing the security within a virtual data
centre solution is more complex than simply testing a standalone appliance.
head2right With Spirent’s new Avalanche Virtual traffic generation technology we have been
able to create a test bed that is fully capable of testing a hybrid physical/virtual
data centre solution – the first of its kind.
head2right Our test bed consisted of a Spirent Avalanche 3100 appliance and a virtual
Avalanche appliance – purely software-based – sitting on a Virtual Machine under
VMware. In this way we were able to test both physical and virtual environments,
independently and in tandem.
head2right We put the new technology to the test using HP TippingPoint’s Secure
Virtualization Framework (TP SVF) – itself claiming to be the most comprehensive
solution developed explicitly for a virtual environment. The SVF solution combines
both physical and virtual elements, working both outside of, and within, a
virtualised environment, so appears ideally suited – on face value – to a virtual
data centre deployment.
head2right We found that; firstly, we were able to successfully test the security of a virtual
data centre and that; secondly, we were able to successfully secure a virtual data
centre with the TP SVF solution.
head2right Using a range of tests that spanned physical and virtual elements of both the data
centre and the test equipment itself, we showed the TP SVF solution to be capable
of preventing a wide range of threats in both the physical and virtual elements of
the data centre, while under full load.
head2right We showed that; at all times, even when the physical components of the TP SVF
were bandwidth saturated, the TP solution was able to maintain communications
between its physical and virtual components and successfully block traffic at the
virtual level.
head2right In conclusion it means that vendors can now provide tried and tested, secure
virtualized solutions to their customers without fear of a security failure.
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 7
Our test environment consisted of a Spirent Avalanche 3100 and Spirent Avalanche
Virtual (running on a VM) as the physical and virtual components of the test, connected to
the HP TippingPoint SVF solution via a 3Com Ethernet switch. The SVF consisted of a
physical IPS appliance, plus software elements running on VMs under VMware, the
vController redirector, the SMS Management Server and Reflex Systems management
software, a 3rd party bolt-on.
We ran literally hundreds of iterations of different tests including several blended
performance + attack tests, such as the one below where we ran a purely virtual (VM to
VM) test and peaked on available bandwidth within this server environment – sustaining
that bandwidth successfully with no dropped packets, while attacking the HP TippingPoint
vController with a CodeRed variation.
The HP TippingPoint SVF successfully blocked all 1432 attacks we generated during an 80
second test period.
Secure Virtual Data Centre Test
8 © Broadband-Testing 1995-2010
INTRODUCTION: THE ISSUES OF SECURING A VIRTUAL
DATA CENTRE
As if it’s not hard enough securing a physical network infrastructure and the data centre
that sits behind it…
Let’s face it – web threats and related attacks are at an all-time high. Global cybercrime
now produces a new variant of malware every 1.5 seconds and literally millions of new
dynamic links are created each day. Each day, companies are seeing more than 50,000
new web threats attempt to overrun their networks – and security vendors are seeing
even more.
When is there a day when some new threat isn’t grabbing the headline?
And now we have the virtual world to protect and the added complexities that brings –
securing the Hypervisor, securing from VM (Virtual Machine) to VM, securing the physical
to virtual connections and vice-versa.
Figure 1 – Virtual Data centre OS - VMware
And on it goes, with one new variation after another. In the context of a medium
business, say, this is a frightening enough prospect, but at the data centre level it
becomes ultra critical.
“Security attacks are increasingly targeted and financially driven. As enterprises move
production applications with their rich information to virtual environments, they represent
an increasingly attractive target for cybercriminals” said Paula Musich, senior analyst with
Current Analysis, adding: “As more mission critical data moves into a virtualized
infrastructure it is imperative that virtual environments meet the same stringent security
requirements as their physical counterparts.”
Worse still, we know there are vendors out there promoting their virtual data centre
solutions without any real way of pre-testing these solutions. So how do they know they
really work? Good question. That’s why our test equipment partner here, Spirent
Communications, has introduced a virtual appliance version of its Avalanche traffic
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 9
generator, so we can test directly within that virtual environment, as we are doing here
with the HP TippingPoint Secure Virtualization Framework or SVF.
The timing is excellent here – Spirent has the first virtual test bed product, HP
TippingPoint the first solution aimed specifically at securing the virtual world. So we are
good to go with the first real examination of securing a virtual data centre.
Figure 2 – HP TippingPoint SVF in Data Centre Deployment
From what HP TippingPoint has seen with its customers, many early virtualization projects
were limited in scope. However, as the footprint of virtualization within the data centre
expands, customers are demanding a unified solution that provides that ability to enforce
security policies regardless of infrastructure type. Moreover, the virtual data centre has to
provide all the security associated with the physical equivalent, maybe more, as there is a
healthy amount of paranoia generally associated with the “less visible” elements of a
network and data storage.
The company sees many organizations now migrating their data centres to virtualization
in order to take advantage of its promise of improved efficiency and cost savings. Some
of these advantages include: system consolidation (including shared storage), reduced
space requirements, reduced power and overall hardware requirements, quicker
application delivery, improved disaster recovery, and better overall management
flexibility.
These advantages cannot be realised if the virtualized infrastructure is at risk of data
compromise, does not meet security regulatory requirements, or is at risk of downtime
caused by cyber threats. Moreover, enterprises want solutions that are trusted and have
been proven to be effective over time for large enterprise protection. After all, the
virtualized infrastructure and its applications are subjected to the same threats that
Secure Virtual Data Centre Test
10 © Broadband-Testing 1995-2010
impact the traditional data centre. So as organisations accelerate the migration of
production workloads and mission-critical assets to the virtualized infrastructure, security
requirements become a strategic element of their migration plans.
Hence the reason for - and purpose of - this test. And, while the testing is focusing on
securing the virtual data centre, performance levels – both at virtual and physical level –
cannot be ignored. That way we are able to measure the impact that virtualisation has (or
not) on data centre performance as well as whether or not it creates a greater potential
security risk.
From the user perspective, the aim here is to effect a valid (read: scientific, repeatable)
simulation of the user experience. Users are no longer prepared to put up with poor
application performance, excessive latency and especially fears about security. This is
critical in the development of virtual data centres; any negative performance and security
aspects immediately invalidates the very concept as users will simply find different
hosting etc options. In other words, weaknesses of any kind as a result of virtualization
are not an option.
Virtual Data Centre Testing: The Key Requirements
What are the basic requirements for virtual data centre testing?
Clearly, in order for us to rigorously test a virtual data centre solution we need to test
each and every element of that solution with a focus – depending on the element under
test – on
- Performance (outright throughput and application availability/latency)
- Scalability
- Security Efficiency
Spirent itself has also based the methodology behind the development of the virtual
Avalanche test software on a very similar concept – PASS (Performance, Application,
Scalability, Security) – so the synergy is complete in this case.
The testbed is designed to run currently at Gigabit and 10GbE (in the physical space) test
options, so we can combine speeds and feeds as in a real (physical or virtual) data
centre. By taking some of the “classic” elements of network testing, then adapting them
to the virtual environment, we create what is effectively a new blueprint for virtual data
centre (or enterprise network) testing.
Our focus in this report is on testing the security of that virtual data centre. In addition to
seeing how the virtual environment performs it is equally of great interest to see how the
virtual environments actually perform in comparison with the physical alternative and in
concurrence. Welcome to the new world…
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 11
DEFINING OUR VIRTUAL DATA CENTRE TESTING: THE
BIGGER PICTURE
The precise components of a “virtual data centre solution” are currently an inexact
science, but there are many commonalities between proposed solutions to date.
Here’s an example of a data centre in the US – interesting because it’s solar powered, so
there’s “green” for you.
Figure 3 – A Solar Powered Data Centre In The US
Secure Virtual Data Centre Test
12 © Broadband-Testing 1995-2010
While this represents a fairly typical data centre, our virtual data centre has more
complexity in terms of more “components”. If we take the diagram below as an example
of the components and approximate topology of a “solution” we at least have a starting
point.
Figure 4 – A Proposed Virtual Data Centre Topology
Here we have shown what is basically a mirrored solution but, in reality, a fully
distributed data centre could have multiple locations. Since one of the key proposed
benefits of a virtual solution is to move away from the ties of fixed locations for data and
processing, a virtual testbed needs to be able to measure performance in a distributed
model. Obviously, specific components/elements of the solution will vary from vendor to
vendor but our test methodology allows for this.
Clearly, as technology moves on, we will continue to refine the methodology with the
usual adds and changes, in order to cover vendor developments. This also includes the
ability to test new technologies, as and when they emerge.
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 13
HP TIPPINGPOINT SECURE VIRTUALIZATION
FRAMEWORK
SVF Overview
With the move to virtualized data centre and enterprise networks, in response to this
need for trusted and proven security solutions specifically for virtualization, HP
TippingPoint has introduced the SVF, aimed at addressing the unique requirements this
change creates.
Figure 5 – HP TippingPoint SVF
At the heart of the SVF solution is the vController. The HP TippingPoint vController takes
advantage of the performance characteristics of the HP TippingPoint N-Platform IPS (see
below) for detecting and stopping threats up to the application layer.
The vController provides a direct path to the HP TippingPoint IPS Platform (appliance) to
inspect and control VM-to-VM communications. Using the VMSafe API, the vController
directs appropriate traffic to HP TippingPoint’s appliance and its threat suppression engine
(TSE) with the aim of providing the high levels of performance and control required in the
virtual data centre. The vController and IPS Platform also operate in unison to support
High Availabilty (HA) capabilities, including fail over of the vController when HA
requirements and configured policy dictate.
Secure Virtual Data Centre Test
14 © Broadband-Testing 1995-2010
Figure 6 – HP TippingPoint vController Environment: As Tested
Part of the SVF solution is the HP TippingPoint Security Management System (SMS) - an
enterprise-class management platform that provides administration, configuration,
monitoring and reporting for multiple HP TippingPoint IPS platforms. Because the HP
TippingPoint SMS provides a scalable, policy-based operational model, it enables
straightforward management of large scale IPS deployments across both physical and
virtualized infrastructure. Additionally, SMS enables separation of duties, which is among
the most critical elements of ensuring secure virtual infrastructure. This is maintained by
providing a unified and centralized IPS management platform for enforcement of security
policies regardless of infrastructure type.
The HP TippingPoint SMS provides granular traffic inspection policy profiles, and enables
inspection of any traffic flows according to user requirements, including VM to VM, and
Host to Host policy configuration. It also enables consistent policy management across
physical and virtualized deployments – from “office-in-a-box” security policy
configurations, where remote sites are set up by combining required apps and other
elements into a virtualized package, to virtualized web apps, to core network
deployments that have their own set of security configuration requirements.
In addition to the security challenges that virtualization presents, one of the greatest
challenges for administrators deploying virtualization is management, visibility and
control of the virtualized infrastructure. Considering the ease of creating and moving
critical applications within a virtualized environment, this challenge is greatly magnified in
the virtualized data centre compared to the physical data centre. Recognizing this need,
HP TippingPoint has partnered with Reflex Systems to bring customers the visibility and
control necessary to effectively configure, manage and control enterprise security
effectively within the virtualized data centre. This is in addition to the HP TippingPoint
SMS, which provides a valuable tool for configuring security policy management,
monitoring and reporting.
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 15
This granular security management combined with the integrated Reflex vCenter and
available Virtualization Management Center (VMC) delivers optimal flexibility, visibility
and control for comprehensive virtualization security management.
HP TippingPoint’s integration with VMware’s VMsafe APIs via Reflex System’s vTrust and
Reflex’s Virtual Management Center (VMC) provides the following features:
head2right Automatic discovery and graphical mapping of virtual infrastructure topology
head2right Supports Separation of Duties (SOD) between operations and network/security
teams
head2right Security teams can monitor vSwitch and VM changes to identify tampering or
disablement of security controls
head2right Upgradeable and compatible with full Reflex VMC
head2right Complete visibility and control over entire virtual Infrastructure
VM Isolation And Segmentation
In the physical data centre, it is a common practice to segment or isolate application
servers used in sensitive areas such as HR applications, CRM and their associated
database servers. Likewise in the DMZ, systems are isolated, like Web applications and
their associated database servers, DNS, e-mail ,etc. When these systems are virtualized,
the need for isolation remains; in fact, the need may be greater since these machines
now share the same hardware resources. HP TippingPoint enables traditional networks to
be segmented by putting physical appliances between them and, with the N-platform,
allows hosts to be segmented without putting an appliance in front of every connection.
By utilizing the VLAN translation features of the new platform, IPS inspection can be
consolidated for a number of system segmentation boundaries.
After enabling host isolation centrally, HP TippingPoint applies that same isolation model
to the virtualized data centre. By tapping into the VMSafe API, HP TippingPoint is able to
isolate VMs within a host or across hosts using either the vController to offload inspection
and enforcement, or vIPS for inspection in the virtualized infrastructure. All three
methods, including VLAN translation for host segmentation; VController for off-loaded VM
and host segmentation; or vIPS for locally executed VM and host segmentation, provide
the benefits of stopping threat cross-contamination among data centre assets, enabling
enterprises to meet security mandates and regulatory requirements such as PCI-DSS.
Secure Virtual Data Centre Test
16 © Broadband-Testing 1995-2010
HP TippingPoint 5100N Intrusion Prevention System (IPS)
For the physical IPS element of our testing we used HP TippingPoint’s 5100N IPS.
Figure 7 – HP TippingPoint 5100N IPS
The 5100N IPS platform is built around what HP TippingPoint calls an Extensible Security
Framework.
This has a modular software design to support faster uptake of new IPS filter packages,
security services and 3rd party integration, such as customer-defined IP reputation
services, the HP TippingPoint Reputation Digital Vaccine (DV) Service, HP TippingPoint
Web Application DV, data leakage protection (filter sets), location-based policies
(perimeter, core…) and customer developed filter sets using the HP TippingPoint Custom
Shield Writer (CSW).
The 5100N uses the latest HP TippingPoint Threat Suppression Engine (TSE), which is
designed to with a combination of pipelined and massively parallel processing hardware,
the TSE is able to perform thousands of checks on each packet flow simultaneously at
Layers 2-7. The TSE architecture also enables traffic classification and rate shaping.
Sophisticated algorithms baseline “normal” traffic, allowing for automatic thresholds and
throttling of malicious or unwanted application traffic.
The HP TippingPoint 5100N ensures that network traffic always flows at wire speed in the
event of network error, internal device error, or even complete power loss. If an internal
error is detected, HP TippingPoint can automatically or manually fall back to a simple
Layer 2 device, configurable per segment.
Additionally, HP TippingPoint offers a Zero Power High Availability (ZPHA) option. In the
event of full power loss to the IPS Platform, the interfaces can switch over to the internal
(where available) or external ZPHA relay allowing traffic to pass unimpeded. For stateful
network redundancy, two HP TippingPoint IPS Platforms can be provisioned using
redundant links in a transparent High Availability mode.
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 17
PUT TO THE TEST
Test Bed Overview
Figure 8 – Our Test Bed (Overview)
Our test bed consisted of a Spirent Avalanche 3100 and Spirent Avalanche Virtual
(running on a VM) as the physical and virtual components of the test, connected to the
HP TippingPoint SVF solution via a 3Com Ethernet switch.
Figure 9 – Spirent Avalanche 3100
Secure Virtual Data Centre Test
18 © Broadband-Testing 1995-2010
The SVF consisted of a physical IPS appliance, plus software elements running on VMs
under VMware, as follows:
head2right HP TippingPoint vController
head2right HP TippingPoint SMS Management Server
head2right Reflex Systems management software
The hardware, including the rack servers hosting the virtual environment, were connected
initially via gigabit connections to the switch. As a separate test (see later) we also
created a connection between the Avalanche 3100 and HP TippingPoint 5100N at 10Gbps.
We were keen to examine both the physical (external) and virtual (internal) elements of
the solution under test and especially the combination of the external and internal
elements since this is how the HP TippingPoint SVF actually functions, combining its IPS
appliance with the virtual controller and management capabilities, as does the Avalanche-
based test bed itself.
In summary then, from a performance perspective we wanted to examine:
head2right External to External performance.
head2right External to Internal performance, availability and scalability.
head2right Internal to Internal performance, availability and scalability
Tests were run with both GETs and Posts, with file sizes ranging from 64bytes to 100KB.
Our results focused on three file sizes – 1024/1200bytes, 10KB and 100KB – in order to
generate suitable throughput levels based around typical real-world usage.
We also looked to then carry out a series of security threat tests, as follows:
head2right Internal to internal with attacks at internal line rate.
head2right As before but with different threat variants.
head2right Adding the internal firewall on the HP TippingPoint vController using a blocking
rule based on a range of client IP addresses, in addition to the threat detection.
head2right Combining external load (bandwidth saturation at the HP TippingPoint 5100N)
with internal threat detection/redirection.
In every test case, several iterations of each test were run in order to guarantee accuracy
and repeatability of results. Additionally, before testing with devices inline we ran back-
to-back tests in order to ensure that the test bed in “vanilla” format was running at line
rate, again to ensure accuracy in our findings.
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 19
Our Test Findings
External To External: Basic Throughput
Though not the focus of this particular report, we wanted to get a baseline performance
figure for all elements of the test bed, including the maximum realistic throughput of the
HP TippingPoint 5100N appliance under sustained attack.
HP claims a maximum sustained throughput of around 5Gbps so, for this test, we directly
attached a Spirent Avalanche 3100 to the 5100N via a 10GbE connection. Note – the
devices were also connected into the main test network via 1Gbps connections to the
virtual elements of the test bed.
Figure 10 – External Tests: Throughput Levels of 5100N
As we can see from the graph, in our tests the 5100N was able to sustain around 4.5Gbps
(4.37Gbps + 135Mbps) while blocking threat traffic in addition to our test http traffic
carrying a 100kbps GET instruction.
Performance Testing From External (Physical) To Internal
(Virtual) Environments
We also wanted to check performance when generating traffic to move from the physical
to the virtual elements of our test bed, the latter obviously being the limiting factor in our
test setup.
Testing with a combination of http GETs and POSTs, using file size variations from
500bytes to 1200bytes, we found that we could sustain between 600-700Mbps with
approximately equal loads of bi-directional traffic using a POST request and a 1200byte
payload size.
Secure Virtual Data Centre Test
20 © Broadband-Testing 1995-2010
Figure 11 – External To Internal Tests: Peak Throughput Levels
So, creating http test traffic with a 1200byte POST request, we were able to achieve just
over 640Mbps of sustained traffic (308Mbps + 333Mbps). Again, this is not wholly
relevant to the primary aim of our testing, but gives us a much needed baseline figure to
work with.
Below, our table shows the different throughput figures recorded with each of the three
different packet sizes, along with the TPS (Transactions Per Second) figures achieved.
External To Internal
Tests GET GET GET POST POST POST
Packet size (bytes) 500 1024 1200 500 1024 1200
Incoming (kbps) 280730 433723 451023 251535 304764 308076
Outgoing (kbps) 77205 69064 62883 294369 333936 333234
Transactions Per Second 49189 43779 39860 47901 32915 28370
Performance In The Virtual Environment
We were very keen to see if the vController redirector in the SVF would impact on
performance and, if so, to what extent. So we created another test using the Avalanche
Virtual to generate http traffic to see what level of sustained throughput we could
achieve, given the fundamental limitations of some elements of the virtual test bed such
as the servers themselves.
Each packet carried a 100KB GET request in order to generate significant throughput with
a view to testing the vController, without creating a bottleneck elsewhere in the test bed.
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 21
We saw peak throughput just short of 700Mbps (690Mbps combined bandwidth).
Figure 12 – Internal Tests: Maximum Throughput
We also wanted to check performance of the vController in terms of any potential
overhead when it was enabled. So we created back-to-back performance tests to run in
the virtual environment, first with the vController redirector disabled, then rerunning the
test with it enabled. So, in both cases we directed traffic at the vController but first
without the redirector being enabled – so traffic would pass straight-through – and
secondly with the redirector enabled so it would be reading every packet as it passed
through. In order to ensure that the test did not trigger any other potential bottlenecks,
but still carried enough traffic to provide a real test, while using the same 100KB load file,
we did change the profile of the test slightly, so that it loaded up more gently and peaked
slightly lower.
Figure 13 – Internal Tests: Throughput With Redirector Disabled
Secure Virtual Data Centre Test
22 © Broadband-Testing 1995-2010
With the redirector disabled we saw peak throughput of around 548Mbps (combined
incoming and outgoing).
Note – in these bandwidth tests we are placing the ceiling where lost packets/failed
transactions start to occur, indicating a bottleneck that is not necessarily due to the
device under test. So in all cases, we are looking at either absolutely zero failed
transactions or effectively zero statistically – i.e. less than 0.001%.
On enabling the redirector we did see this throughput fall very marginally to around
540Mbps (combined incoming and outgoing), which is effectively insignificant in the real
world .
Figure 14 – Internal Tests: Throughput – vController Redirector Enabled
However, we would expect to see performance improve significantly with a higher
performance server. Indeed, the server is the fundamental limiting factor for vController
performance we feel during these tests. With the redirector now enabled throughout the
testing, we wanted to see how performance varied with different file sizes, as used
before, only purely with a GET action this time around, but with far smaller packet sizes.
Internal To Internal Tests GET GET GET
Packet size (bytes) 500 1024 1200
Incoming 64404 111952 126393
Outgoing 17831 17886 17715
Transactions Per Second 11348 11342 11205
We found that maximum throughput, without incurring significant packet loss, was
unsurprisingly far lower than that achievable with a 100KB size as transactions per
second – peaking at 11,348
across the tests.
External To Internal: Scalability
In order to gauge application availability under load, and therefore, potential scalability
based on increasing server power, we measured the server response time during all
performance tests.
External To Internal Tests
Packet size (bytes)
Server Response Time
(ms)
The results were excellent
the SVF solution could easily scale with more powerful servers for the virtual end and
more bandwidth would not appear to be an issue.
Scalability/Application Availability In The Virtual
Environment
Internal To Internal Tests
Packet size (bytes)
Server Response Time (ms)
Repeating the GET tests within the virtual environment we got more excellent results;
well sub-millisecond in each case. So the scalability appears to extend fully
of VMware and VM-based security.
Figure 15
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
500
ms
Secure Virtual Data Centre Test
© Broadband-
– became the limiting factor, but performance was consistent
/Application Availability
GET GET GET POST POST
500 1024 1200 500 1024
0.324 0.297 0.265 0.311 0.374
– sub millisecond in every case. The implication here is that
GET GET GET
500 1024 1200
0.58 0.564 0.811
– Internal Tests: Scalability – Server Response Times
1024 1200
0.58 0.564
0.811
Packet Size in bytes
al
Testing 1995-2010 23
Ap Av
POST
1200
0.475
Re ;
into the world
Secure Virtual Data Centre Test
24 © Broadband-Testing 1995-2010
Security: Attacking The Virtual Environment
Having established performance and latency levels that we were very hap
the test bed in use, we then looked to see how the SVF solution would cope with a series
of different attacks. We began with an internal (virtual environment only) test using the
Avalanche Virtual targeting the VController directly with a co
generation with a 100KB file size and injected threat traffic, based purely on a CodeRed
variant in this instance.
Figure
While generating around 430Mbps of general http traffic over 90 seconds we also injected
1,432 CodeRedII attacks. All were successfully blocked with no reduction in overall
throughput from the general data stream.
py with, given
mbination of http traffic load
16 – Internal Tests: Security – Code Red Test
Server Response time average was 1.72ms.
II
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 25
We then repeated the exercise but with a multi-threat attack, based on a number of
different stateful attacks that we knew were contained within the HP TippingPoint device
signature database and therefore should be able to successfully block. What makes this
theoretically more difficult than normal, of course, is that we are attacking across a
saturated link in the virtual environment – VM to VM – so the test is really on the
vController’s ability to successfully block while being effectively max’d out, traffic wise.
For this test we ran over a more extended period – 10 minutes – injecting threats 80
seconds into the testing. This enabled us to see how the vController coped with an
extended attack and how that attack impacted on any other element of performance.
Generic http traffic with a 100KB file size was generated, after which we injected a
selection of stateful attacks and checked what the block success rate was at the end of
the test.
Exploitation Results
Attack Success/Failure @ Client/Server
Attack Name Attack Type Client Success Client Failure Server Success
antvilleXSS.xml Stateful 0 2752 0
coderedII.xml Stateful 1 2751 0
core_mailslot_dos.xml Stateful 0 2752 0
ms05-051.xml Stateful 0 2752 0
WebDAV_rs.xml Stateful 1 2751 0
Figure 17 – Internal Tests: Security – Multi-Threat Attack
While generating over 520Mbps of bidirectional traffic, we injected over 13,000 attacks.
Throughput remained constant throughout the test, though average server response time
did increase to 12ms.
We saw two attacks pass through the client side but these never reached the server, so
all the above threats were successfully blocked. When you consider that, at no time was
regular network traffic passing through VM to VM impacted upon at all, this is a very
creditable performance.
Secure Virtual Data Centre Test
26 © Broadband-Testing 1995-2010
To date, while testing the IPS element of the SVF solution, we had only enabled the
firewall on the vController with a simple rule. So we decided to rerun a multi-threat test
but add a rule to the firewall to block a batch of clients based on IP address – 20 in total,
out of a pool of 254 clients.
Exploitation Results
Attack Success/Failure @ Client/Server
Attack Name
Attack
Type Client Success Client Failure Server Success
coderedII.xml Stateful 0 2736 0
lupper1.xml Stateful 1 2735 0
lsass.xml Stateful 0 2736 0
nimda1.xml Stateful 0 2736 0
nimda17.xml Stateful 3 2733 0
putty_bof.xml Stateful 0 2736 0
zotob.xml Stateful 1 2735 0
Figure 18 – Internal Tests: Security – Multi-Threat Attack Plus Firewall Filter
As before we checked the threat attack success rate. At the client end five attacks passed
through but, again these never reached the server, so were nullified successfully.
In terms of the firewall filter, we were looking for 20 out of 254 clients to have their
traffic blocked throughout the test. If we take this as a percentage of the total traffic, we
find that the results showing that 7.9% of URL transactions (our http traffic) were blocked
equates almost exactly to our desired block rate.
Note: All threats contained within the HP TippingPoint signature database were
successfully blocked within our test definitions. Those that were passed were shown to
not be current in the database used for the DUT (these could be added by HP
TippingPoint).
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 27
External And Internal Combined Traffic/Attacks
For our final attack test, we wanted to additionally load up the 5100N appliance, thereby
combining 10Gbps traffic directly between the Avalanche 3100 and the 5100N and gigabit
traffic within the virtual environment with a multi-theat attack on the vController. Since
the vController redirects to the 5100N, this would test the capabilities of the SVF to
continue to work successfully when max’d out in every element of the solution.
Exploitation Results
Attack Success/Failure @ Client/Server
Attack Name Attack Type Client Success Client Failure Server Success
coderedII.xml Stateful 770 595 0
lupper1.xml Stateful 746 619 0
lsass.xml Stateful 291 1074 0
nimda1.xml Stateful 753 612 0
nimda17.xml Stateful 733 632 0
zotob.xml Stateful 581 784 0
coderedII.xml Stateful 536 829 0
lupper1.xml Stateful 574 791 0
lsass.xml Stateful 208 1157 0
nimda1.xml Stateful 572 793 0
nimda17.xml Stateful 581 784 0
zotob.xml Stateful 461 904 0
Figure 19 – External-Internal Test: Security – Multi-Threat Attack with 10Gbps Traffic
Generating around 4.5Gbps again between the Avalanche 3100 and the 5100N IPS we
created an extended set of threat attacks aimed directly at the vController once again.
On this occasion we found a higher level of threat attack success at the client side but still
none were able to get through to the server without being blocked. So even when some
attack packets are missed at the client side – as is truly not uncommon - they are not
able to manifest themselves at the server.
Secure Virtual Data Centre Test
28 © Broadband-Testing 1995-2010
SUMMARY & CONCLUSIONS
With Spirent’s new Avalanche Virtual traffic generation technology we have been able to
create a test bed that is fully capable of testing a hybrid physical/virtual data centre
solution – the first of its kind.
We put the new technology to the test using HP TippingPoint’s Secure Virtualization
Framework (TP SVF) – itself claiming to be the most comprehensive solution developed
explicitly for a virtual environment. We found that; firstly, we were able to successfully
test the security of a virtual data centre and that; secondly, we were able to successfully
secure a virtual data centre with the TP SVF solution.
Using a range of tests that spanned physical and virtual elements of both the data centre
and the test equipment itself, we showed the TP SVF solution to be capable of preventing
a wide range of threats in both the physical and virtual elements of the data centre, while
under full load.
We showed that; at all times, even when the physical components of the TP SVF were
bandwidth saturated, the TP solution was able to maintain communications between its
physical and virtual components and successfully block traffic at the virtual level.
Overall we are satisfied that the HP TippingPoint SVF solution can provide a secure virtual
data centre and that it is designed to scale well.
Finally it is important to note that the Avalanche Virtual test bed itself is more than
capable of testing in a secure virtual data centre – and, in theory, every component of
that virtual data centre. Importantly, it has taken away the need for guesswork when
deploying virtual data centres – now you can pre-test, so there are no excuses for
vendors out there...
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 29
APPENDIX 1: THE VIRTUAL DATA CENTRE COMPONENTS
WE CAN TEST
Back-End Storage
Depending on the solution proposed by the vendor this may vary in terms of both the
technology and the topology.
SAN – Storage Area Networks; the classic solution as sold by the likes of Brocade and
now Cisco. Here we have to allow for various flavours of technology as the original, pure
Fibre Channel solution has expanded to include various flavours such as:
- FCIP (Fibre Channel over IP)
- IPFC (IP over Fibre Channel)
- FCoE (Fibre Channel over Ethernet)
- iSCSI protocol – often now favoured in SAN solutions
NAS – Network Attached Storage; originally meant for low-end solutions but now being
proposed by the likes of Isilon Systems as a true alternative to SAN at any level. Typically
uses a combination of Ethernet (gigabit or 10GbE) and Infiniband for external/internal
bandwidth.
Blade Servers – Typically the server element of the solution will be in the form of blade
servers, regardless of OS. What we need to take into account is the virtualisation of the
servers, and the virtual OSs:
head2right VMware (in its various incarnations
head2right Citrix Xen
head2right Apple Parallels
head2right MS Virtual Server
Load-Balancers
Most data storage subsystems at the date centre level will feature some form of load-
balancing, typically with an external device, such as an F5 Big-IP. These normally sit
directly in front of the server farm, sometimes within the blade servers as an additional
blade and even as a virtual application in their own right.
We need to be able to test every variation on a theme here.
Data Centre/Core Switch
Here we are looking typically at hybrid 10GbE/Gigabit Ethernet switches with varying port
densities and chassis formats. Regardless of the vendor and port configuration, the test
basics are fundamentally the same however.
Secure Virtual Data Centre Test
30 © Broadband-Testing 1995-2010
Security Devices
Typically a data centre may now have a tiered security configuration consisting of multiple
security devices. In some cases these are also placed both internally and externally
(WAN/Internet facing), so we should allow for configurations that require a doubling-up
and test accordingly.
Most common components are:
IDS/IPS
Realistically we are now looking at IPS dominating the market and IDS as history. In
some cases the UTM (Unified Threat Management) appliance, incorporating IPS is
positioned more upmarket, but typically these are more low-end devices that won’t
feature in a data centre solution. So we are primarily concerned here with standalone IPS
devices.
Firewalls
As above – firewalls appear in many multi-function security appliance, but beyond that
even they are becoming more specialised, not just stateful versus stateless, but
application layer versus network layer etc, so we need to allow for testing of all the
different offerings that might make up a vendors’ data centre solution. Here, particularly
between basic packet/network layer firewalls and application (layer 4-7) firewalls that
include deep packet inspection, the test requirements are very different.
Routers
Where the virtual data centre solution under test involves a distributed element, via a
WAN/Internet connection, we need to include the router provided as part of the solution
within our tests. Testing here would focus along the same “linerate” lines as with the
switches under test, albeit with typically asymmetric bandwidths between internal and
external connections.
WAN Optimisation Devices
Where WAN links are involved, now this will typically involve additional technology to the
router itself in the form of WAN optimisation/acceleration/packet shaping devices. Here
we need very specific test plans depending on the specific device under test as the
technology varies enormously between the different vendors.
Secure Virtual Data Centre Test
© Broadband-Testing 1995-2010 31
APPENDIX 2: SPIRENT AVALANCHE VIRTUAL
Figure 20 – Spirent Avalanche Virtual
The Spirent Avalanche Virtual is the industry’s first virtual infrastructure and application
test solution for cloud services and the virtual world. It is designed as a high performance
virtual appliance to test in the private, public, and hybrid clouds and is based on the
Spirent Avalanche hardware platform. The virtual appliance works directly in conjunction
with Avalanche hardware and appears as an Avalanche hardware resource to the Spirent
Avalanche Commander console manager.
Currently it supports the VMware ESX 4.0 virtual server but support is designed to be
broader in the future. Avalanche Virtual is designed to test a number of different
elements within a cloud or virtual infrastructure, such as:
head2right What is the maximum number of open connections and what is the impact of
connections per second on bandwidth?
head2right How do network and application events correlate under network congestion?
head2right Does the virtual firewall ensure that infections or malware does not proliferate
from one VM to another?
head2right Does a virtual load balancer provide full availability and maximum performance of
virtualized applications?
head2right Can virtual appliances perform at line-rate?
head2right How effectively can the blade servers support multiple applications requiring large
memory?
