Most software companies have considered automated testing and many have fully automated their regression test cases in an effort to reduce manual effort needed to test new builds of their software. Many companies that have been successful with automation attribute it to keyword driven testing techniques that reduce the time spent creating test cases.
This newsletter addresses why companies consider automation and best practices for ensuring that time spent automating test cases provide a return on investment.
Why Automate Your Test Cases?
Many companies run their regression test cases manually, so when does it make sense to begin automating your regression test cases? It makes sense to automate your test cases when you can no longer run the regression test cases on each build created. For example, if you are doing daily or weekly builds of your code to quality assurance and you cannot quickly run your regression test cases with each build, it is time to consider automating them. Automating your test cases provide these benefits:
- Quicker Releases – By having your regression test cases run automatically, your software quality team can concentrate on testing new features of your software and less time regressing existing features.
- Higher quality releases – Your software releases will have fewer bugs and require less customer support because they will be of higher quality.
- Happier Customers – Your customers will be happier and more willing to serve as testimonials for future prospects.
Why Does Automation Fail?
Many companies are thinking about test case automation or have experimented with it in the past. Some companies that experimented with it eventually abandoned it because of these reasons:
- Poor Understanding of Test Automation – Many companies see automation as a silver bullet that will allow them to quickly automate every test scenario quickly and allow them to abandon manual testing and reduce staff. In reality, automated testing is designed to quicken the running of regression test cases but it is not a substitute for manual testing. A quality-oriented software team will see the value of utilizing both automated and manual test cases to ensure great test coverage and higher quality releases.
- Improper Tester Education – Automation requires a tester to learn the testing tool. Companies that purchase the tool but not any training will eventually abandon the tool because the testers are not equipped to use the tool.
- Lure of Record / Playback – Many companies think that they can quickly get up and running with automation by simply recording their screen actions and playing them back. While record and playback will create scripts that they can use as a starting point for your automation, it does take time to update the scripts to be more re-usable and it takes scripting language knowledge to do this.
What is Keyword Driven Testing?
Most automated tools require the test engineer to understand a scripting language (VB Script, Java Script, etc.) to write their automated test cases. Most tools have the ability to create the scripts using record and playback, but this does not always write the most efficient scripting code and is not as re-usable and maintainable.
Since many testers do not have deep scripting skills, it is imperative that your automated testing tool has a way to create Keyword Driven Tests. Keyword Driven Testing is a way to define automated test cases without the need for scripting skills. It allows a tester (or even a subject matter expert) to create automated tests by describing each step of the automation.
For example, if you are automating the login process of your application, your user will access your application, type in their user-id and password and press a button to login. Traditionally, testers would do this by writing VB Script that will navigate to your application, identify each object on the screen (user-id, password and login button), then write script to enter in the user-id, password and to press the login button.
With keyword driven testing, the tester does not need to understand the scripting language to make this happen, they can simply describe the event (navigate to your application, enter in "abc" for the user-id, enter in "xxx" for the password, press the Login button when done). As you can imagine, this is a much simpler approach to automated testing than scripting.
How have Successful Companies Implemented Automation?
Successful companies understand the enormous benefits of automated testing and have implemented strategies to ensure that they receive the maximum return on investment with their test efforts. Below are the secrets to becoming successful with test automation:
- Start Smart – Automation efforts are similar to software development efforts -- it takes upfront thought and a good test design architecture to ensure that your automated test cases will be re-usable and easy to maintain. Before jumping into automation, ensure that your automated tool has the ability to maximize your efforts with keyword driven testing -- as this will ensure that you can easily re-use your automated test cases.
- Start Small – Start your automation efforts on an established project that already has a good set of manual regression test cases written. Take those manual regression test cases and develop your automated test cases that will replace them. Once this is done, you will find that you can automatically run those automated test cases each day of builds and with minimal efforts, freeing your team up to do more manual exploratory test cases. Once you have this established for a single project, move on to other projects and expand your effort.
- Blend Automated and Manual Tests – Not all test cases should be automated. Some test cases require a human eye to ensure that screen cosmetics are appropriate, that data is reasonable, etc. Spend time automating test cases that do not require this type of human contact and continue to use manual test cases when appropriate. A good application lifecycle tool should allow you to track both automated and manual test cases.
- Keep track of Metrics – Keep track of how many automated and manual test cases are run for each build, how many pass, how many fail, etc. Track how many additional automated test cases your team can write with each release so that you can determine average time needed to develop automated test cases in the future. Track how many defects are found post-production to determine if your automation efforts are paying dividends. A good application lifecycle tool should allow you to track metrics for both automated and manual test cases.
- Build on your Successes – Once you have successfully implemented automated testing on a single project, roll it out to more projects. Help other teams in your organization learn the benefits of automation and help them do it right.
- Schedule Your Automation Efforts - A good application lifecycle tool will allow you to organize your automated test cases into "test sets" that allow you to run test cases in a specific order. It should also allow you to schedule those test sets to run at specific intervals (nightly, weekly, monthly, etc.).
How can I learn more about Test Automation?
There are a number of automated and manual test management solutions on the market, as long as your selected solution allows you to follow the best practices of this newsletter, you can easily begin making the transition to automation and begin receiving a return on investment from your efforts.
If you don't have a solution for both automated and manual testing, Software Planner (http://www.SoftwarePlanner.com) and TestComplete (http://www.testcomplete.com) offer an integrated solution that offers both automated and manual test cases and has a keyword driven testing engine that ensures re-usability and maintainability.
Helpful Resources
Below are some helpful resources and templates to aid you in developing software solutions:
Posts
