post-img

NFV Testing and Validation challenges

Posted by Hans Ashlock February 27, 2015
NFV Testing and Validation challenges

Technology manufacturers and service providers plowing their way into NFV testing and validation are looking to affirm the agility that NFV offers, but between them and that promise lies the challenge of characterizing performance in virtual environments. One of the key advantages of NFV is being able to replace dedicated appliances with bare metal servers, strategically placed in data centers and running intelligent software. However, the shift from physical to virtual infrastructure necessitates a fresh approach to monitoring, analytics and DevOps orchestration.

NFV service assurance challenge: Ensuring performance in network operations
With NFV, the method of delivery for network services is changing in a big way, while end user expectations for high-quality, reliable service are staying mostly the same. Accordingly, although organizations may implement NFV with the primary goals of trimming costs and gaining agility, they must ensure along the way that their uptake of low-cost, flexible servers and software does not compromise network performance, which is essential to the value proposition of NFV orchestration.

Basically, for the end user, it has to seem like nothing has changed, even though so much actually has changed in the data center. The streamlined infrastructure within virtual environments must deliver services with quality at least on par with their legacy physical counterparts. One key to ensuring that NFV is up to the task requires dealing with  the monitoring challenges that emerge as dedicated appliances become less prominent.

"In a virtualized network environment, assurance solutions and processes must transition from traditional, reactive monitoring to proactive, real-time intelligence and analytics, tightly integrated with and coupled to the network and services, as well as orchestration and policy systems," observed Sterling Perrin, Senior Analyst at Heavy Reading, in a January 2015 white paper. "Assurance solutions must also span multiple services, including Ethernet/IP, video and mobile."
  
Teams have plenty of incentives to switch to NFV - such as being able to run CPU-intensive functions that have low throughput and economically scale their services - so the overarching issue here isn't so much motivation as it is execution, as Perrin notes in his emphasis on particular types of monitoring solutions. There are other obstacles to take into consideration on the road to NFV too, such as vendor resistance to the commoditization of the network hardware business, but those potential problems are macro considerations that network engineers can't as easily influence.

NFV service quality challenges: Ensuring performance before service release
Test environments for NFV allow for a lot of flexibility. Compute resources can be scaled up and down depending on the type of testing being performed and environments may be copied from development or production. At the same time, all elements of the NFV test environment, including the NFV infrastructure as well as the virtualized network function that run on top of them, should overall be as similar as possible to the characteristics of the production environment. For example, VNF instantiation should be in line with the hypervisors and compute, storage and network resources available in production.

NFV testing may be conducted at multiple, geographically dispersed locations. For this reason, using virtual appliances is often preferable to relying on physical appliances. Moreover, they are cost-effective and offer equivalent capabilities in most scenarios, with physical appliances only really having an edge for cases requiring optimal data plane performance and microsecond-level timing.

When attempting to simulate real-world traffic conditions, it makes sense to figure out the production bush hours traffic rate and then come up with a characteristic mix of client and server activities. The BHTR figure can then be used a baseline for different tests. For instance, 5 percent of BHTR would be effective for functional testing, with 150 percent of it might be workload scenarios such as auto-scaling. Provisioning, scheduling instantiation latency and virtual machine stall are good metrics for NFV service testing, while forward rating and frame loss are ideal for data-plane performance evaluation.

NFV can increase the complexity of a networking environment by enabling agile service chaining that runs on VMs rather than dedicated appliances. Automating this setup requires a platform that is adept at handling both legacy and SDN/NFV assets. QualiSystems TestShell allows for an object-based approach, in which automation is broken down into small-scale objects that lend themselves to high reusability. Objects can be easily segmented into NFV management and orchestration, NFVI, VNFs, legacy network services and test appliances, making it possible to address any type of infrastructure during testing and validation.

The takeaway: NFV has tremendous promise to create more open, interoperable, efficient and cost-effective network infrastructure. Challenges are ahead, particularly with ensuring adequate service quality and performance. NFV DevOps orchestration solutions like CloudShell can help service providers build a cloud-enabled foundation for automated testing that ensures NFV service chains provide the quality of experience users expect.

CTA Banner

Learn more about Quali

Watch our solutions overview video