Posts

HP UFT/QTP vs. Selenium – Automated Test Tool Comparison

HP Quick Test Pro, also known as HP Unified Functional Testing, is currently a well-known force in the web-based testing market, but Selenium is quickly gaining mindshare and advocates as a more capable, open-source competitor. There are several clear-cut distinctions between each test tool that should make choosing one or the other a straightforward decision in most cases.

Comparison Highlights Between QTP and Selenium

Cost – QTP requires a license fee for acquisition and more fees for upgrades and add-ons, but Selenium is a totally free, open-source download and will always be so.

Testing Applicability – Selenium is only for testing web-based apps, whereas QTP can test client-server and desktop applications in addition to web-based apps.

Cloud Ready – QTP’s one-script/one-machine execution model cannot make efficient use of distributed test execution via cloud-based infrastructure. Selenium-Grid is specifically designed to run simultaneous tests on different machines using different browsers and different operating systems in parallel. Thus, it is a perfect match for cloud-based testing architectures and services.

Execution Efficiency – QTP tests one application per machine, whereas Selenium can execute multiple, simultaneous tests on a single machine. Furthermore, QTP script execution takes more RAM and CPU power than does Selenium. QTP can run in multiple Windows VMs, but these are more resource-hungry than Linux VMs, which Selenium can utilize.

Browser Compatibility – QTP works with four browsers, albeit some of the most popular ones, whereas Selenium works with those plus five more browsers of which two are headless. Headless browsers provide additional test execution efficiency.

Language Compatibility – QTP tests are written in Microsoft’s VB Script. Selenium tests can be written in one of nine different popular languages including Java, JavaScript, Perl, Python and PHP. Thus, you can adapt it to the programming resources you have on hand.

OS Compatibility – Selenium tests applications on all major OSs including Windows, Linux, OS X, Solaris, iOS and Android. QTP runs only on Windows.

Support – Selenium is supported by a vibrant, active user community, but it does not have dedicated support staff, whereas you can buy technical support for QTP. For some issues, paid support provides faster resolution cycles.

Technical Challenges When Using Selenium

Although Selenium has a number of meaningful advantages over QTP, including its much broader compatibility with different app and test configurations, it is not without technical limitations:

  • Selenium is not uniformly compatible across browsers. It is most compatible with Firefox, so scripts developed for Firefox may need tweaking to run in IE or Chrome.
  • It has no object types other than WebElement or Select, and no repository for storing object mappings.
  • There is no inherent support for data-driven testing.
  • HTML tables and other elements require coding to work correctly.
  • Image-based tests are harder to accomplish.
  • Dialog box support is limited.
  • In general, Selenium requires a higher level of coding skills

Some of these issues are resolved via testing frameworks compatible with Selenium. However, these may add upfront development effort when developing or integrating with such a framework.

Tips on Migrating from QTP to Selenium

When migrating from QTP to Selenium, take a piecemeal, trial approach to get started. Choose a script language with which the team has the most experience already, or one that best supports the type of automation framework you plan to build.

However, at the beginning, defer building a framework and simply write a handful of simple Selenium scripts. Collect notes about issues as you go along. Incorporated those learnings into building larger test suites. It is easier to write Selenium scripts from scratch than trying to port them directly because of the many differences between QTP and Selenium.

Once you become familiar with Selenium and commence building an automation framework, look at tools that play well with Selenium such as Maven for compilation, Test NG to drive unit, functional and integration tests and Jenkins or Hudson if you plan to use a Continuous Integration methodology.

If your web-based automation test projects require the highest flexibility across browsers, OSs and script languages at the lowest cost, then Selenium is the clear winner. However, especially in the earliest stages of adoption, more skilled programming resources are required for script generation and integration into an automation framework of your choice.

The many pros and cons between QTP and Selenium may present difficult choices if you currently use QTP, but otherwise Selenium provides the better payback and versatility over the medium-term.

Functional Test Automation Using HP QTP And QC – Part 1

HP_QTP Functional Test Automation Using HP QTP And QC – Part 1

Lessons from a HP Partner in Vancouver

Part 1 of this series covers best practices for automating functional testing using HP QTP. Part 2 of the series covers managing functional testing with QC.

Before I delve in, let us first take a deep breath and appreciate the implications of Testing in our daily lives! A 30 second Ikea commercial nicely summarizes the importance of quality.

 

QTP is a tool for automating functional and regression tests. QC works as a quality management platform that offers requirements management, test management, defects management, and business process testing. Together they are a powerful force which significantly facilitates functional test automation.

These are award winning products, now an integral part of the multi-billion dollar HP Software Portfolio.

This is a 2-part series. In this article, Part 1, I am going to focus on QTP and describe some best practices to automate functional tests. In Part 2, I will shift my focus to the functionality of QC.

I’m currently involved in a large automation project. These are our views based on our project experience.

Part 1: HP QTP

Steps & Guidelines

1. Obtain knowledge and define test scope

Step 1 is to get familiar with the application under test, test environment and manual test steps. We should consider the following areas when we acquire the knowledge of the test: test purpose, test environment, applications required, test verification, and unexpected behaviours.

With the knowledge to understand and execute the test, we will have enough information for automating the test and defining the test scope.

2. Write manual test steps

Even though the test is automated and manual testing becomes a very rare case, it is good to have clear and precise manual test steps for reference. When we write the test steps, we should assume the readers have very limited knowledge and experience of executing this test. Therefore, the steps should be very specific and detailed. For example, pressing TAB may not be documented sometimes because it seems too tedious, but it may be a very critical step to the test.

3. Divide the test steps into components

A test may consist of 20-30 test steps and require multiple applications. Since some test steps are used in lots of test cases, grouping test steps into components can exploit the reusability of the test steps. There is no strict rule for dividing the test steps into components, but each component should serve one purpose and can be reused as a standalone component in lots of tests.

4. Write test script for each business process test (BPT)

After the test steps are grouped in several business components, we can start developing test scripts.

Here are some steps for developing a script:

i. Create a shared object repository:

The object repository contains the objects used in the test. This repository can be built by simply recording the steps with QTP or using QTP Object Spy. After the objects are added to the repository, we can save this repository in the QC server, and this repository can be used among different tests.

ii. Create an application area:

Every BPT test script is associated with one application area, so we need to create a new application area with the required add-ins, an object repository, functional libraries, and recovery scenarios. The test script inherits these settings from the application area. An application area can be used by different tests, but a test can select only one application area.

iii. Develop the test script:

QTP allows users to write the test script in keyword view or expert view. If complicated logic is involved (such as if-else, while loop, randomization, etc), keyword view is not able to provide such support.

Developing the test script includes:

  • Record a simple test flow: Recording the test steps with QTP is a good starting point; in fact, do not bother too much about handling special cases, adding verifications, and so on at this stage. This phrase can identify any problems or issues with running the application with QTP. In some cases, QTP does have problems with launching and running some applications. After we know of these problems, we have to first tackle these blockers by coming up with some workarounds.
  • Add verification: There are two ways to verify data; we can either use a QTP checkpoint or use a customized checkpoint. QTP provides 8 types of checkpoints, but sometimes creating checkpoints for parameterized data is not straightforward. On the contrary, a customized checkpoint provides the flexibility to verify in different situations. Here is an example of creating a customized checkpoint:

If <Condition> Then

Reporter.ReportEvent micFail, “Login”, “Username is invalid.”

End If

  • Implement logic: Recording the test steps does not generate the important logics behind the scene; therefore, we have to implement some logics for the test by ourselves. The logic can help handling special scenarios, generating test data, verifying complicated elements, and so on.
  • Parameterize variables: In most cases, we have to parameterize variables so that the test script can be generic and accept different data. Parameterization using QC and QTP is straightforward.

5. Combine components into flows and a test

After the test scripts of the business components are complete, we can combine them into flows and/or a test in QC.

In Part 2, I’ll delve into QC and eventually tie-up all the pieces together to expand on how the combination of QC & QTP fits into the overall strategy.

For more details, please feel to connect with me at Maggie.yu@optimusinfo.com

Performance Testing Tips and Tools with HP LoadRunner

picplz_20110429_00001874417_00001-300x225 Performance Testing Tips and Tools with HP LoadRunner

Performance testing in Vancouver

Last night at VanQ, our technical manager Larry Ng presented on performance testing tips and tools using HP LoadRunner (full video embedded below). He explained the performance testing process from design to execution and reporting and how HP LoadRunner is utilized for performance testing.

From his experience he found that although automation tools like HP LoadRunner are expensive, they are worth the money in time saved. Specifically, he noted that HP LoadRunner has been drastically improving the amount of time he spends designing and running tests as opposed to scripting.

Here are  a few key tips for running successful performance tests:

  1. Schedule a dry-run prior to running tests so that all participants are coordinated and familiar with the entire process.
  2. Take a snapshot of databases prior to running the tests so it’s easy to start fresh for the next test.
  3. Reboot the servers between tests and clear the cache in browsers (if applicable).
  4. Use an instant messenger client to keep everyone organized and more importantly, keep a timed log of events.
  5. Use Perfmon to monitor server health throughout the test. Use the IM log to match up the times.
  6. Choose test scenarios that are stable and heavily used.
  7. Group your test scripts into transactions so that entire process is measured (ie: The whole login process instead of each step.)
  8. Use a naming convention to keep your transactions organized. This helps with both reporting and script maintenance.

Here are a few helpful links that we discussed for those that want to learn more about HP LoadRunner:

At Optimus we work with clients on a variety of platforms. Here are a few of the other leading performance testing solutions:

Commercial Tools:

Open-Source Tools:

And of course, as promised here are the slides from Larry’s presentation. We are uploading the video as well – I will post it when it’s available.

Visit the OptimusQA performance testing page for more on our testing services.

Performance Testing using HP LoadRunner

hploadrunner-300x243 Performance Testing using HP LoadRunner

HP LoadRunner

Join us at VanQ this Thursday to discuss Performance Testing using HP LoadRunner. Our own technical manager, Larry Ng, will be sharing his hands-on experience using real-life examples.

If you haven’t made it to a VanQ event yet, this is your chance. VanQ hosts a free event each month in Vancouver to discuss software testing best practices. We regularly sponsor the event as it brings in excellent speakers and a strong network of local software testing professionals.

The event starts at 6:45PM on Thursday, April 28, but come early to meet local testing experts.

To RVSP, please visit VanQ’s website.

Hope to see you there!

Quick Test Professional 11 Review

QTP-11-177x300 Quick Test Professional 11 ReviewQuick Test Professional (QTP) is an automated functional Graphical User Interface (GUI) testing tool created by the HP subsidiary Mercury Interactive that allows the automation of user actions on a web, client based, or desktop application. QTP can be integrated with Quality Center and the automated test cases suite can be executed directly from Quality Center.

Quick Test Professional 10 came with exciting new features like Local System Monitoring while Running Your Tests, Versioning, QTP 10 integration with QC, and Call Actions Dynamically during the Test Run. People from the test automation community were expecting new features in QTP 11 like the long awaited support for Firefox, functional library load at run time, and identification of objects using XPATH property.

Finally QTP 11 is out with following exciting new features!

Regular Expression Evaluator:
The use of regular expressions in the dynamic object properties has been simplified using new Regular Expression Evaluator. For applications which require a lot of regular expressions in automation, upgrading to QTP 11 will definitely help in simplifying things and solving complex puzzles.

Visual Relation Identifier:
After upgrading to QTP 11, automation engineers that were working on QTP 10 or previous versions will see a new property in the Object Repository Visual Relation Identifier. This feature helps to identify objects based on the relationship with other objects in the application under test.
Earlier we had to write logic to find objects in the application. This was then used during execution of automated test cases as most of these types of objects have an index value associated with them. Now using the visual relation identifier we can identify objects based on their relationships with other objects. The scenario above is just one example of how this feature can be helpful in identifying other complex objects in the application.

Enhanced Result Viewer:
QTP_11-Results-Viewer1 Quick Test Professional 11 Review

The Result Viewer has a completely new look as compared to Result Viewer in QTP 10. The new Result Viewer displays summary data and pie charts for current and previous runs with quick links to open results for the previous runs. The Run Time Data Table also has enhanced look which is now displayed under the executive summary section.

Load Function Libraries at Run Time:
Like the ‘LoadAndRunAction’ feature which was added in QTP 10 (which we used lot in long END-to-END automation scenarios), QTP 11 has taken the next step in this direction which makes scripts more dynamic. This new functionality allows run time loading of the functional library using the ‘LoadFunctionLibrary’ statement.

Firefox support:
QTP 11 now supports Firefox 3.6; however, you have to install a patch before you can automate on Firefox. In our experience, Firefox 3.6 support by QTP 11 works pretty well.

Automatic Parameterization:
Now we can instruct QTP to automatically parameterize input data by enabling this feature during recording. If you do a lot of parameterization in test scripts, this will save time.

Silverlight Support:
Automation support for Silverlight is available in QTP 11. The Silverlight add-in is displayed in the add-in manager after upgrading QTP from previous versions. One good thing is that the Silverlight add-in is provided by HP, not like the flex add-in that is provided by Adobe. So we can expect better support for Silverlight applications by QTP.

Extensibility Accelerator:
Using the Extensibility Accelerator we can design automation support for custom developed web-controls which are not supported by object recognition mechanism of QTP.

XPath and CSS:
Like Selenium, now QTP can also recognize objects using the XPath and CSS properties of the objects in the application.

Integration with Service Test:
Service Test scripts can now be called in QTP scripts. This helps in automation of scenarios which include SOA service steps along with GUI functional steps in one script.

Conclusion:

As per our experience, the upgrade from QTP 10 to QTP 11 is a painless transition. With the features mentioned above, like support for Firefox, Visual Relation Identifier, Regular Expression Evaluator, and the enhanced Result Viewer, it’s worth upgrading your existing automation suite. QTP 11 can now be adopted for automation of Silverlight application. If you’re already using QTP 10, then the upgrade to QTP 11 is free of charge and definitely worth the effort.

If you’d like help setting automated software testing with QTP 11, please give us a call or send us an email at info@optimusinfo.com.

Software Testing HP User Group

software-testing-hp-user-group-vancouver-300x72 Software Testing HP User Group

The software testing HP User Group.

Optimus is proud to partner with HP in hosting Vancouver’s HP User Group. With a focus on the latest solutions coming from HP, these user groups are a great forum for learning advanced techniques and how to make the most of the latest functionality.

We work with clients in Vancouver and abroad to implement HP (and other) software testing solutions. We have an experienced team that has planned, implemented, and analyzed large software testing projects using both waterfall methodologies and agile software testing.

In our own development work we have a strong focus on software testing. Before we publish any code changes we go through a combination of automated and manual software testing to ensure our standards are met. Our own development is generally more agile than traditional as we push out updates frequently and test constantly. Our clients are pleased with this methodology because they see a constant stream of new features while they can manage and steer their product design.

Below is a video of Shane Evans, Solutions Architect at HP, discussing the benefits of adopting a framework technology. This video is from the latest HP User Group held in Vancouver, BC.

For those that joined us, thank you for participating and sharing your experiences, we hope you enjoyed the dinner and learned some helpful techniques. For those that missed it, keep an eye on our blog, Twitter, and Facebook as we’ll mention the next date as soon as possible.