Test Automation Benefits and Challenges

Time to market is often an overriding concern of software development and with good reason. If your organization delivers a new software product or a revision to an existing one later than promised, there are significant negative side-effects. Not the least of these is a dilution of publicity efforts to build anticipation in customers for the new product. Obviously, the customers themselves will be disappointed also.

The Importance of Testing to Software Schedules

Software schedules always seem “doable” at their start, but as they progress slippage typically occurs. A key reason for this is that essential testing of the product, including test requirements, does not commence simultaneously with code development. This situation leads to finding showstopper bugs in the code or design late in the development cycle. These take more time to rectify than if they were found earlier.

Test Early and Often, Automatically

Pushing an inadequately tested product out the door in order to meet the schedule is not a viable option either. A perception of poor quality in the market will linger far longer than the disappointment of a late release that is of decent quality.

Thus, thorough testing is a key requirement for any software project. It must be begun early and to ensure effectiveness and efficiency, it must be automated as much as possible.

What Can Be Automated?

Automation can be effectively applied to nearly every aspect of the QA process:

  • Unit tests within developers’ IDE
  • Functional testing of software components and libraries
  • Negative testing that attempts to force errors through invalid inputs
  • Stress testing for determining functional capacity of the software and platform
  • Performance tests to measure compliance with product requirements
  • UI/UX tests or instrumentation to determine usability of the software
  • Regression tests to detect new defects in existing code at each new build
  • Test case and test script generation to improve test coverage

Automated Testing Benefits

Relative to software schedules, testing automation has a key advantage: it saves time, which means it reduces the risk of schedule slip, improves quality and reduces short- and long-term costs. The greater the automation effort, the more advantages that accrue.

Faster Test Case Generation and Better Coverage

Developing test cases is often a time-consuming process done manually by translating design and code requirements into test scripts. Most test cases can be generated by tools that utilize code annotations, execution path analysis, randomization of inputs or script generation from recording user interactions. All test case generation methods increase testing coverage.

Improved Reliability and Repeatability

Automation eliminates mistakes that arise from manually running tests, such as intentional or unintentional omissions in test steps, which miss bugs or produce ones that do not actually exist. When bugs are detected, for the sake of developers, they must be reliably repeatable. Automating bug reproduction increases the likelihood that they are.

More Efficient Workflow

Comprehensive test tools allow testers to automatically select individual tests or test suites, record the results and take appropriate steps in the case of defect detection. These steps include logging technical details, screenshots, videos and generating bug reports. These automatically generated data can be shared instantly with both test and developer teams.

Enabling Agile Methodologies

Reducing temporal lag throughout the software development process is an essential feature of agile methodology. The closer in time that requirements, design, coding and testing can exist, the faster iterative development can take place. This leads to higher work efficiency among all teams involved in producing the software. Automated test tools, especially those with superb collaboration capabilities, are essential to quick turnaround in defect detection and removal.

Saving Time on the Next Project

The ability to store tests in scripts or code is not only a time saver for the current project. These can be reused for the next revision of the product or even for different but similar products undertaken later.

The Challenges of Test Automation

Automation is certainly not a substitute for poor testing practices within the organization. In fact, automating bad behavior is likely to make things worse. Additionally, automation cannot alter the “physics” of testing. It cannot guarantee the absence of bugs nor is it a guarantee of a quality product release.

If test automation is new to an organization, the QA department has hurdles to clear:

  • Unrealistic expectations are likely to exist regarding how much time will be saved in the testing schedule
  • It costs time and money to analyze, acquire, learn, utilize and maintain automated test tools
  • A new level of expertise may be required within the QA team for writing automation code
  • Test software is like any software, it has bugs and these must be dealt with

Test automation may require additional compute and network infrastructure.

Some, but not all of these challenges, can be mitigated if the organization employs a cloud-based testing service in lieu of building all the software and hardware infrastructure in-house.


The simple answer in support of test automation is that it saves time. Despite the initial costs, once established, the return on investment is rapid. QA staff and developers become more effective and quality conscious in an environment where both are able to collaborate efficiently. Testing schedules have higher predictability and a higher quality product, delivered on time, is more likely.

Quick Test Professional 11 Review

QTP-11-177x300 Quick Test Professional 11 ReviewQuick Test Professional (QTP) is an automated functional Graphical User Interface (GUI) testing tool created by the HP subsidiary Mercury Interactive that allows the automation of user actions on a web, client based, or desktop application. QTP can be integrated with Quality Center and the automated test cases suite can be executed directly from Quality Center.

Quick Test Professional 10 came with exciting new features like Local System Monitoring while Running Your Tests, Versioning, QTP 10 integration with QC, and Call Actions Dynamically during the Test Run. People from the test automation community were expecting new features in QTP 11 like the long awaited support for Firefox, functional library load at run time, and identification of objects using XPATH property.

Finally QTP 11 is out with following exciting new features!

Regular Expression Evaluator:
The use of regular expressions in the dynamic object properties has been simplified using new Regular Expression Evaluator. For applications which require a lot of regular expressions in automation, upgrading to QTP 11 will definitely help in simplifying things and solving complex puzzles.

Visual Relation Identifier:
After upgrading to QTP 11, automation engineers that were working on QTP 10 or previous versions will see a new property in the Object Repository Visual Relation Identifier. This feature helps to identify objects based on the relationship with other objects in the application under test.
Earlier we had to write logic to find objects in the application. This was then used during execution of automated test cases as most of these types of objects have an index value associated with them. Now using the visual relation identifier we can identify objects based on their relationships with other objects. The scenario above is just one example of how this feature can be helpful in identifying other complex objects in the application.

Enhanced Result Viewer:
QTP_11-Results-Viewer1 Quick Test Professional 11 Review

The Result Viewer has a completely new look as compared to Result Viewer in QTP 10. The new Result Viewer displays summary data and pie charts for current and previous runs with quick links to open results for the previous runs. The Run Time Data Table also has enhanced look which is now displayed under the executive summary section.

Load Function Libraries at Run Time:
Like the ‘LoadAndRunAction’ feature which was added in QTP 10 (which we used lot in long END-to-END automation scenarios), QTP 11 has taken the next step in this direction which makes scripts more dynamic. This new functionality allows run time loading of the functional library using the ‘LoadFunctionLibrary’ statement.

Firefox support:
QTP 11 now supports Firefox 3.6; however, you have to install a patch before you can automate on Firefox. In our experience, Firefox 3.6 support by QTP 11 works pretty well.

Automatic Parameterization:
Now we can instruct QTP to automatically parameterize input data by enabling this feature during recording. If you do a lot of parameterization in test scripts, this will save time.

Silverlight Support:
Automation support for Silverlight is available in QTP 11. The Silverlight add-in is displayed in the add-in manager after upgrading QTP from previous versions. One good thing is that the Silverlight add-in is provided by HP, not like the flex add-in that is provided by Adobe. So we can expect better support for Silverlight applications by QTP.

Extensibility Accelerator:
Using the Extensibility Accelerator we can design automation support for custom developed web-controls which are not supported by object recognition mechanism of QTP.

XPath and CSS:
Like Selenium, now QTP can also recognize objects using the XPath and CSS properties of the objects in the application.

Integration with Service Test:
Service Test scripts can now be called in QTP scripts. This helps in automation of scenarios which include SOA service steps along with GUI functional steps in one script.


As per our experience, the upgrade from QTP 10 to QTP 11 is a painless transition. With the features mentioned above, like support for Firefox, Visual Relation Identifier, Regular Expression Evaluator, and the enhanced Result Viewer, it’s worth upgrading your existing automation suite. QTP 11 can now be adopted for automation of Silverlight application. If you’re already using QTP 10, then the upgrade to QTP 11 is free of charge and definitely worth the effort.

If you’d like help setting automated software testing with QTP 11, please give us a call or send us an email at

Agile Software Testing and Automation

Automation can be important to agile software testing because the development process is much more iterative and requires frequent testing. A waterfall development cycle only requires major testing at the end of the development process; whereas, agile development requires testing throughout.

What’s the difference between waterfall and agile software development?

Below is a typical example of the waterfall methodology. As illustrated, the process leads developers through a series of sequential steps with major testing done towards the end.

Waterfall Software Development Cycle

waterfall Agile Software Testing and Automation

The agile methodology is much different than the waterfall methodology. Agile development requires very active project management. The process begins with planning, but then instead of building an entire application at once, agile development builds each functional component separately in what are called Sprints. The illustration below shows how the process constantly involves end-users, business analysts, and end users.

Agile Software Development

agile_testing Agile Software Testing and Automation

Agile software development is a flexible process that rapidly builds each piece of a software project, adding functionality every few weeks until the entire project is complete. This kind of rapid development has the flexibility to meet the demands of evolving business requirements.

Agile development teams are often broken into small groups that can work on individual pieces of the overall project. Agile development requires organized communication between business analysts, programmers, and end users in order to ensure all business requirements are being met.

For example

For example, imagine two companies building an email program. Company A uses the waterfall methodology, so they create a requirements document of what their email client will need to do. They list everything from sending and receiving mail, to managing an address book, to scheduling appointments. Then they begin development and build the entire email program over a few months. Once the entire program is built, they begin testing. Often using a combination of manual and automated testing, they go through the program with a fine toothed comb and identify issues. Then they go back to the program and remedy all the bugs before releasing it to end users.

Company B uses an agile development process, so they get a team started right away on basic functionality. In the first week they build the framework and test the user interface. The following week they create the ability to receive emails. The week after that they build the ability to send emails. This continues week after week. They build a small piece of the program, test it, and release it to end users. Their team has already released the program to the client and is adding functionality every week. This gives Company B the ability to remain flexible while responding quickly to user feedback.

Where does testing come in?

In the waterfall development methodology, testing is at the end. Once the project is complete, the entire program is tested using a combination of automated and manual testing. A large list of test cases is worked through to ensure proper functionality of the entire program.

In the agile development methodology, testing is conducted throughout the process, often on a daily basis. That is why agile software development requires increased automated testing because the testing is much more frequent. It would be nearly impossible, or at least in-efficient, to manually test an entire program on a daily or weekly basis throughout the development process. In that way, automated testing makes it possible because it can run through a large series of test cases rapidly, efficiently, and most importantly, effectively.