Test Automation – Challenges overlooked

                                   Test Automation – Challenges overlooked

“To improve is to change; to be perfect is to change often. “-Winston Churchill

This is exactly what the software industry must do to be perfect and thereby succeed. But what is also important is that, the changes reach their customers fast enough and with best in class quality. Continuous deployment and continuous delivery attempts to do exactly this. In fact, in continuous delivery model, as soon as the code is written, it is integrated, tested, built and configured. So now the million-dollar question is that how do we ensure that testing also happens with the same speed? The answer of course is automation. But while we decide on automation, there are lot of challenges that we tend to overlook to achieve this speed.
➢ Maintainability of Automates
With the speed at which we create automate, sometime we overlook the fact that these automates must be maintained in the future. What most of the testers have in mind, is to automate fast, preferably record and playback and to look for a tool which is easy to learn. Sometimes in this race, we forget that these automates must run for long- which means that these scripts should be maintained for quite a long duration.
➢ Focusing only on the test case flow
Most automates, consider test case as their bible while developing scripts. The main and crucial flow is of course automated. But some of them at least tend to forget about negative tests. We must remember that value of a test comes from the kinds of inputs provided and not just the quantity or frequency of the tests executed. Test engineers must be able to find a balance between scripting difficult scenarios and achieving test coverage.
➢ Sticking to one tool
Test automation tools are decided by the test organization by considering many factors like application technology, cost, ease of learning etc. But it is a hard truth that there is no one tool to suffice all testing requirements like UI, Database, functional, security and performance. But due to the various constraints listed above, test engineers narrow down to use only one or maximum two tools. This usually creates a major roadblock, when we have applications with mix of technology. Here the usage of more than one tool could be a solution, but an expensive one.
➢ Having a robust automation framework
Test Automation framework is your overall game plan It is a set of guidelines for creating and designing test cases. It could include coding standards, test-data handling methods, object repositories, processes for storing test results, or information on how to access external resources. In the race for achieving speed and quantity, test engineers sometimes forget the importance to build a robust framework for automation. This could lead to higher maintenance, reduced reusability of scripts, more efforts for reporting etc. Test teams spend lot of time and effort to identify a perfect tool or tools for automating their applications. But to get the best out of those tools, you need to have a proper automation framework which helps you to properly organize and run your whole automated suite.
I am sure most of them would agree that, they have overlooked at least one of those above. It is difficult to address all these challenges during our automation project, but not impossible. With careful planning and a futuristic thought process this can be achieved without compromising on the speed and quantity. In fact, we can even have more efficient, faster and perfect automation suite.

Cross Browser Testing in the Cloud

Anyone who has tested web application would understand the importance of testing a web page’s compatibility with different browsers and OS combination. This is a very important but time-consuming and costly affair. Considering the fact that there are hundreds of browser/platform combinations to test, you can very well imagine the infrastructure required to cater those needs. It is here that the cloud services come to our aid.

There are different cloud services which offer the multiple platforms that we require for our tests. We can choose one based on our requirement and perform our tests. In this blog I would like to give an overview of how we can do it.

There are many cloud services like TestingBot, Sauce Labs, etc. that offer cross browser testing solutions. Some of them offer automation tools while some offer manual testing platform.

Why choose cloud?

Cross browser tests demand heavy infrastructure, due to the need of different browser and OS combination and also different versions of the same browsers. Sometimes, it is a challenge for testers to downgrade the browser versions to check downward compatibility of particular software. These challenges can be handled by using clod services.

By using for a particular cloud service provider you can choose from a list of browsers available to test your website .Please see the figure below a screenshot from Sauce Labs:

Testing our applications on mobile is yet another challenge. With mobile technology galloping, the challenge of the test management team is to take into account of the Wirth’s Law. Cloud service providers are trying hard to tackle this problem faced by organizations.

By using the service provided by the cloud, you can perform your testing on any of the browsers manually. See fig below the screenshot for testing a website in Linux android 4.0:

The manual tests you perform are recorded and can be revisited at a later point in time.

For those who have difficulty in getting their selenium configuration done and then later maintaining the software, you can use the selenium offering by any particular cloud for your automation activities.

Challenges

Whenever you choose a cloud service provider, it is important to consider the performance of the cloud, so that the speed of your tests does not get affected. Also security of your data is of a major concern when you use a public cloud. If you intend to encrypt the data before sending it to the cloud, it means addition expenditure which has to be taken into account.

And most importantly the reliability of the Service provider is major concern for organizations. An unannounced disruption of service can nullify all the benefits that it offers.

Wrap up

So in view of the above mentioned points, cloud is well suited to browser testing, provided we do it in the right way.  After an analysis of the requirement (like infrastructure) of the company carefully, a test strategy has to be put in place. As the second step, a service provider has to be identified and then the tests have to be executed.

SOFTWARE TESTING USING CLOUD

Testing is an integral part of the software development life cycle and no product can be released to the market without testing it efficiently. Organizations often face various challenges like budget, cost for test, geographical distribution of users, deadlines, infrastructure etc. during the test phase. Cloud based testing could be one of the solution for these problems. In this blog I would like to explain how.

There has always been confusion between cloud testing and testing using cloud. Cloud testing is testing cloud applications but testing using cloud refer to tests which use the infrastructure on cloud for software testing.

Selecting the cloud service provider

Once the task of designing the test cases is completed, it is time to select a service provider. There are certain points that need to be kept in mind while choosing the service providers.

•  The service provider’s configuration model
•  The cost
•   Service Level Agreement
•   Disaster recovery Plan
•  Security

Considering all the above points, you choose a service provider that best suits to your need and budget.

Setting up the infrastructure

After you decide on the service providers, it’s time you take a look at the infrastructure. There are certain crucial factors that need to be addressed when you decide on the cloud infrastructure.

•   Reliability and availability
•   Integration with the existing management tools
•   Good User interfaces
•   Scalability

Once you are sure you have everything in place it’s time you leverage the cloud servers and then start testing.

What kind of tests can be done on cloud?

Most of the conventional test activities can be done on cloud. But the most prominent one would be performance testing due to the high infrastructure it demands. Functional tests can be also be done on cloud especially automation and cross browser tests of web based applications. You can use the cloud services for acceptance testing and system integration testing as well due to the real time environment that it offers.

Challenges that need to be tackled

The lack of universal standards in terms of security, pricing etc. pose a big challenge for organizations when they chose cloud service providers. Also using the cloud services without proper knowledge and planning can sometimes incur additional costs, especially if you are opting for the pay per use policy.

Wrap Up

To summarize, testing using cloud can be beneficial to organizations if planned properly and executed carefully. Jumping to a big scale cloud based testing can be sometimes disastrous. Taking small steps first and scaling up would be the ideal process for the adoption of cloud based testing.