Spirent circle logo

Testing Challenges and Strategies for Successful SASE Deployments


SASE emerged in 2019 as a visionary security framework where security functions are hosted in the cloud to address security challenges resulting from the rapid transition to a remote, hybrid workforce – but not without challenges. Learn how to overcome these challenges and operationalize SASE network architectures and services running on these networks.

Secure Access Service Edge (SASE) converges networking and security domains into a distributed cloud environment, where it connects and secures any IT resource—physical, virtual, and mobile—with unified management and policy. This dynamic, policy-based approach delivers the agility, elastic scaling, and resiliency digital enterprises require, achieving significant advantages over VPNs deployed in a traditional centralized data center. As technologies converge, there are new challenges to overcome on the road to delivering the hybrid distributed systems that intelligent operations demand.

New challenges on the road to distributed network security

Key operational challenges for SASE deployments

1. SASE assurance

Managed Service Providers (MSP) are obligated to deliver comprehensive service-level agreements (SLA) to their enterprise end-users. However, unlike network/infrastructure SLAs, the SASE KPIs have not been standardized, varying widely for each specific service and application. At the same time, SASE testing standards are just beginning to emerge. Establishing a methodology to validate SASE end-to-end behavior, especially considering the many proprietary SASE-based CNFs, NFVI variations, and lack of purpose-built tools, can be expensive and time-consuming.

2. Network functions and applications assurance

To effectively deliver SASE end-to-end service-level management, MSPs must validate each network function that is deployed in the SASE edge cloud. The complexity is exacerbated by the many proprietary network functions, each with its own API and management tool, and the lack of a common assurance methodology. Again, SASE testing standards do not exist yet, and at this early stage there are no purpose-built tools, compelling MSPs and their partners to rely upon home-grown testing tools that are unable to scale.

3. SASE service applications behavior

An important factor in the selection of cloud security controls such as next-generation firewalls (NGFW), web application firewalls (WAF), or secure web gateways (SWG) is the footprint, scalability, and robustness running in various cloud environments. Validating the efficacy of the security controls is essential but requires specialized expertise and realistic emulation of legitimate and malicious traffic profiles to validate effectiveness and performance in conjunction with finding the optimum balance between quality of experience (QoE) and security effectiveness.

4. Security policy and performance assessment

Security rule set validation is imperative for successful rollout and policy configuration for SASE environments. The primary challenge is the ever-changing threat landscape. Assessment of security rule sets needs to be performed on a continuous basis, not at a specific instance in time, to account for new threats and vulnerabilities, evolving policies and network configuration and inventory changes.

5. Zero Trust Network Access (ZTNA) behavior

ZTNA relies on trust brokers to grant access based on identity, policy, and context (vs network connections). MSPs must be able to validate ZTNA elements for scale and sustainable access request rate, while ensuring that the policy criteria are continuously enforced by security controls like NGFW and data loss prevention (DLP) for application and data access. Lack of ZTNA standardization has resulted in proprietary products or services with varying capabilities, making it harder to quantify and contrast different solution options.

Given these various challenges, how do we operationalize multi-domain, distributed, and dynamic SASE deployments? In the next section, we will discuss the principles and test methodologies we leverage to enable our customers to overcome the above-mentioned challenges.

Modern SASE, secured SD-WAN and application security architecture

Testing strategies for operationalizing SASE deployments

1. Test every deployment environment

Considering SASE architectures are multi-domain, hybrid, and distributed, SASE deployments need to be validated end to end – from the remote users and branches through the SASE POPs (points of presence) to end-application servers. SASE managed service offerings may be connecting over many IPS networks and different public and private clouds. Therefore, it is important to characterize the latency and performance (throughput, transactions per second) profiles of these networks and verify whether the SASE architecture is improving or hindering overall performance and latency as it secures these networks.

These principles apply whether you are creating 5G Core or Metro Edge, operating a quality assurance lab, or integrating with CI/CD/CT developer toolchains. Holistic testing and assurance capabilities are essential, regardless of what the architecture is based on, VMs, containers, or bare metal.

2. Select traffic patterns and KPIs according to business needs

Selecting appropriate SASE test traffic patterns that are representative of your network traffic and applications will ensure that performance is characterized under realistic conditions, avoiding the false confidence that comes with simple traffic patterns offered by open-source tools. Testing all security components of the SASE framework, including the network underlay, cloud infrastructure services, and business applications, requires multiple KPIs for all technology and services levels.

Quality of Experience (QoE) is the best unit of measure because it directly indicates end-user satisfaction. It is based on performance, detection of errors, and variability for SSL/TLS based services and overall transactional latency. This metric is also optimal for altering the underlays that application flows will forward across the SASE environment. In addition, bandwidth/throughput, concurrent users and connections, connection and rate are good baseline metrics to help right-size infrastructure to business needs and continuously baseline as part of the CI/CD/CT practice, for a measured approach towards operationalizing the change management process.

3. Identify security and performance vulnerabilities

Besides validating realistic application traffic profile performance and QoE, there is also a need to realistically model threat vectors to validate the security efficacy of the SASE security stack with the latest threat vectors. For this, it is essential to have access to a constantly evolving library of threat content covering vulnerabilities, malware, exploits, and ensuring alignment to security industry framework such as MITRE ATT&CK, to improve the organization’s security posture and bridge gaps in SASE deployments.

To fully characterize the security effectiveness, it is important to emulate hacker-like behavior by using evasion or obfuscation techniques, including cloaking threat vectors in TLS. There is also a need to characterize the impact of security policies on end-user experience by emulating legitimate application traffic at scale in conjunction with malicious threat vectors. If security controls and polices are impacting business-critical activities, the organization will look for ways to bypass those controls, which could adversely affect its security posture.

4. Characterize the impact of Zero Trust (ZT) architecture

Since Zero Trust is a critical component of SASE architecture, it is vital to characterize the scalability of the Zero Trust architecture in terms of the authentication rate for concurrent users that the identity and access management (IAM) can support. In addition, characterizing the effectiveness and impact of ZT policies such as micro-segmentation, lateral threat movement and DLP on performance and end-user QoE is also important.

The need for intelligent automation, reporting and data-driven predictions

Validating multiple levels of cloud and services infrastructure for complex use cases—such as 5G low-latency slicing or 360⁰ security policy enablement for distributed environments—requires an additional level of intelligence, as well as comprehensive automation, reporting and correlated data rather than partial, incomplete information.

Business impact analysis is essential for understanding how new policies or service disturbances will affect your end-customers and SLAs. Root cause analysis supplemented by anomaly detection will provide in-depth analysis, not only for historical data, but also to expose future issues and performance limitations. In short, the massive amount of data generated by hundreds, or thousands of applications requires a profound understanding of the impact of security policies on performance, and predictive analytics to accurately forecast future behavior and potential impact on security policies.

The agile way in which SASE networks and policies are evolving requires validation that is continuous and integrated into operationalizing change management practices to best manage security and performance drifts.

Learn more about the requirements for effective SASE and Zero Trust validation, related use cases and testing strategiesRight arrow icon

Like our content?

Subscribe to our blogs here.

Blog Newsletter Subscription

Sashi Jeyaretnam

Sr. Director of Product Management for Security Solutions

Sashi Jayeratnam is the Senior Director of Product Management for Spirent where she leads the Security Solutions group. She has over 20 years of experience in networking and cybersecurity technologies, and has been instrumental in driving and introducing market-leading application performance and cybersecurity test solutions for on-premise, cloud and hybrid networks. Sashi regularly speaks at security events and webinars on the importance of taking a proactive and measured approach in mitigating cybersecurity risks. Prior to Spirent, Sashi lead Product Management at Keysight Technologies.