How to Accelerate Test Automation Execution Time for Web app with Authentication

Our client is a company that strives to provide constant product improvement to its customers. Part of this process involves gradually optimizing the features of the web applications that the users are working with on an everyday basis. This frequent evolution of the applications increases the possibility of introducing new flaws regardless of the experience of the engineers involved in the development and delivery. Since our client invests in assuring that those mistakes can be avoided as much as possible, our team provides partial automation test coverage for some of the applications, which minimizes the number of potential new issues.

Issue

When the introduction of new features in one of the applications starts to pile up, the number of tests slowly but continuously is increasing. Such an enlarged quantity of test cases can take significant time to execute. Even though their execution is independent (not blocking other processes) and fully automated, running locally or on continuous integration/delivery servers, monitoring of this prolonged process can be extremely difficult.

Approach and steps

To reduce the execution time, our engineers analyzed the automation tests execution pipeline of one of the applications to detect which parts can be optimized.

Step 1: Construct execution pipeline

The current status of the execution pipeline had to be visually constructed so it would be easily understandable by engineers who did not take part in the current solution.

Step 2: Analysis of execution flow

The execution flow is analyzed to detect parts that can be potentially reconsidered. One of those key points was to try to avoid the repetitive authentications on each test, initially done because of the necessity to run a specific test suite (usually defined per framework’s module) or even a subset of tests. Those repetitions consist mostly of calls to the authentication logic each time a set of related test cases started to execute to safely make a manual selection or exclusion of the tests that need to be executed on demand. Therefore all the involved parties agreed that the authentication handling had to be reconsidered.

Step 3: Proposals on optimizing the authentication part of the pipeline

We presented three viable options:

1. Take advantage of the authentication cookies;
2. Make changes in the configuration methods that the testing framework offers;
3. Reconsider the used technology by investigating possible alternatives.

The third option was rejected because it did not represent a short-term solution, it was questionable due to the existence of the alternative technology, and finally, it was without a clear outcome. The first two options were achievable although the second one seemed more appropriate and could be accomplished with less effort. Taking that into consideration, option number two was chosen and put in motion by our engineers.

Solution

The conclusion was to proceed with moving the authentication logic to a configuration method that is executed before all configured test suites. Therefore no matter what kind of test subset is executing, the authentication will be done only once. The related tests that were covering the authentication process were also moved.

Since authentication is one of the most crucial parts of the application, it made sense to leave them as mandatory checks anytime a specific suite ran. Although the visualized pipeline looks more complex by involving one extra step, with the new restructuring, the iteration is gone and the execution inside the test suites is simplified.

Result

The results of the monitoring phase are looking encouraging. This optimization is not only beneficial for accurate testing procedures, but it also has a direct impact on the daily work of the Automation developers.

1. It resulted in a reduction in debugging time.
2. Our engineers successfully managed to accelerate the execution time of all existing test suites by up to 40%.
3. This approach also provided a learning opportunity for the engineers in the form of taking further advantage of the testing framework’s configuration for other purposes and challenges without having to make significant changes in the technology.