What’s so scary about test automation?

March 19, 2013

In every testing environment and QA lab today, management and engineers perform hundreds of thousands of repetitive actions and procedures when testing and validating components and infrastructures. Ongoing proper execution of these actions is essential to ensure the quality of the network and products under test.

Unfortunately, when attempting to automate, most of these procedures are script-based and are difficult for others to reuse. So, every small change in an element or network layout may require editing of the script in multiple locations. When the question of automating these processes arises, more often than not test teams shy away from the subject. "Once bitten, twice shy" equals reluctance regarding the possibility of creating user-friendly, cost-saving automated frameworks for even the most seasoned testing personnel. Past problems could have been:

  • self-developed automated systems that were so complicated that only the developing team could use them
  • "automated" systems that relied heavily on "capture and replay" scripting that required tremendous effort for recording thousands of component-specific scripts.
  • long automation projects that weren't worth the development time expended instead of performing the actual testing work
  • the overhead for adding and maintaining automation of new features grew over time until it was no longer worthwhile to continue to do so on top of the same framework

Despite these obstacles, there are ways to prevail over testing monstrosities and automate all or part of test procedures.

Conquer the fear
Through the development of basic building blocks that are repeatedly re-used to create test automation with little or no need for coding knowledge, test engineers can easily triumph over test automation horrors. A simple "drag and drop" scenario enables test designers to design a logical flow chart and then manipulate actions without having to perform actual scripting. Although the frameworks can be implemented differently, the below guidelines are commonly needed:

  • Test engineers must be able to build the automation on their own without involving code developers.
  • The time to automate testing of a new feature cannot be much greater than the manual testing time.
  • long automation projects that weren't worth the development time expended instead of performing the actual testing work
  • All the automated commands, parsing, reporting, etc. must be easily contained within re-usable and replaceable building blocks from which the automation process can be designed and built top-down.

This means that:

  • Changing a component or a small part of an existing layout will only require changing specific building blocks, not recreating the entire automation from scratch.
  • We can leverage the knowledge of our domain experts by letting others use their complex procedures packaged into simple building blocks.
  • The user interface should not vary regardless of vendor or component. All topologies, devices, interfaces, procedures, tests, regressions, results, reports and dashboards need to be supported within a single integrated platform.
  • Automation generates numerous data that must be easily aggregated from multiple tests, testers and stations and then filtered, grouped and tabulated according to test-specific requirements into a concise report.

Your ‘out of the box' approach needs
For building successful test automation, we discussed the guidelines and now let's define what we are looking for:

  • CUsability. This includes number of users, ease of training and how quickly it can be implemented.
  • Maximum functionality. Ability to support lab management, asset reservation and scheduling, device integration, topology definition and setup, automation development and execution, data collection aggregation and reporting and dashboards.
  • Flexibility.and scalability. Ease to upgrade and integrate new capabilities
  • Strength.of the interface library. The interfaces support all the device and application controls needed and can deal with steps that are not "out of the box.".
  • Vendor. What is the vendor's core business? Does the product have a roadmap, and how aligned is it with your requirements? What support can you expect? And perhaps the most important factor: What references of successful
    automation deployments does the vendor have?.



In summary, test automation doesn't have to be scary. Adopting the right approach can lead to great results.
About the Author

Alex Henthorn-Iwane joined QualiSystems in 2013 as Vice-President of Marketing, and is responsible for worldwide marketing and public relations. Prior to joining QualiSystems, Alex served as Vice-President of Marketing at Packet Design, Inc., a provider of enterprise network management software solutions, and has 20+ years of experience in senior management, marketing and technical roles at both hardware and software startups in the networking, security and telecommunications field.