The Automation paradox:

Reach out to a QA Manager, consultant, or tester and mention software testing and we see a simple nod of acknowledgement. Mention Automation and you can feel the excitement. Automation, although an overused term, is still a very esoteric subject.

So many times we’ve seen companies investing in new tools and investing time in building frameworks. But very often, these initiatives end up as ‘one-hit wonders’. Why? Because the automation objectives are not defined well in advance. QA is not given enough freedom (aka “time”) to identify automation opportunities and familiarize themselves with best practices (scripting and script maintenance nuances).

The Approach:

Prior to any automation project, QA should internally ask themselves the following fundamental questions:

1. Is the current size of the regression test suite large and frequent enough?
2. Will automation cover at least 20-25% of test coverage (% varies based on project goals)?
3. Are there repetitive tests which are frequently being done manually?
4. Can tests be ‘x-driven’ (read ‘x’ as Data, Keyword, Modularity, Model, or Hybrid)?

If answers to all of the above are ‘Yes’, then justification of time and effort in exploring automation is viable.

Incidentally, automation is one of those activities where QA draws attention from management. And we all know whenever the latter is involved, it’s handy to have a business case ready (and you will collect brownie-points).

A simple approach of calculating the automation cost:

1. Automation cost = cost of tool(s) + cost of script creation (labour cost) + cost of script maintenance (labour cost)

2. Automation test execution cost

Automate only if:

• Automation cost + Automation test execution cost < Internal manual cost of executing the same number of automation test cases.

This is probably the cleanest and most simplistic view of automation; however, other priorities (distractions) will always take precedence. This approach will make QA’s job much easier.

Automation, points to consider:

1. We need not automate ‘everything’: For successful automation, identifying the right candidates is the key. Some key parameters to consider are: Script reusability, frequency of test scripts execution, and availability of skilled resources. This last point is the most critical and frequently ignored. If there is an automation strategy, it is imperative to build automation expertise.
2. Maintenance: Probably the trickiest of all the automation issues. If the application has frequent changes, maintenance of the scripts can be a QA’s nightmare.
3. Tools are means to an end: More often than not we try to fit in our needs based on the available features of the automation tools. The tool fitment should be the last stage. The goal should be to ensure our house is in order with the minimum necessities, before we take the plunge.
4. Career Growth: This by far is the most compelling reason to factor in. The repetitive manual testing work can sometimes impediment a QA’s career path. However, on the brighter side, the repetitive work can be ‘boxed’ relatively easily and handed over. This can free up significant time of testers and they can learn new skills. One of our clients started an automation initiative even though it was not needed. Their only reason was to grow their QA team and ensure they constantly acquire new skills.

All said and done, automation does not eliminate manual testing. In fact, manual testing forms the backbone for any QA department. The question to ask is: why are we taking this path and where do we envision ourselves in the next two years?

Automation can reduce your manual testing effort by as much as 80%. But globally, around 60% of test automation projects fail.

If you’ve decided to take the automation route, we recommend you expect around 9-12 months, before you start reaping the benefits.

As a parting thought, we would recommend all QA engineers read the best seller by Seth Godin, “The Dip“.

It is an 81 page pocket book, has no mention of QA / Tester anywhere, but has some powerful lessons which we can implement in any projects, especially automation. The picture below captures the essence of the book.

The Test Automation reward curve.

*Image source: Blog of Ben Nadel

To read more about our Test Automation work, read our latest case study on API Test Automation.

These are some of our thoughts as we continue to work closely with our clients as their trusted technology partner. Any pointed feedback is always welcome; as it would help us further improve our processes. Please feel free to contact me directly at anand.deb@optimusinfo.com

Customized Reports in SQL 2012

In a recent meeting, we had a discussion around the difference between Business Intelligence (BI) reports and canned reports that come with Enterprise Resource Planning (ERP) systems such as Microsoft Dynamics. Similar discussions are now a regular part of technology strategy meetings of many organizations.

This post answers the following question: why are external BI reports still required in addition to reports generated by the ERP solution?

Fig 1. Typical ERP Report

Real Canadian Super Store is a Canadian hypermarket chain. Their ERP would capture countless transactions on a daily basis throughout the country. If a Sales Director wants to see a sales report for April 2012, then the ERP system will generate a report similar to Fig. 1.

A typical comparison report would have multiple dimensions such as, Items, Region, and Timeframe. In this example, the ERP’s canned report does not provide enough information on region-specific sales. Additionally, the sales comparison to previous months is missing.

It is often the case that a canned report meets 90% of an organization’s needs, but that last 10% can provide a great deal of value. With our clients, we have often seen that the last 10% of value is added by bringing in data from other systems.

In the example described here, data from a 3rd party system could be brought in to compare publicly available weather information to product sales. Then the Sales Director will see trends that associate with heat-waves and cold-snaps.

Fig 2: Detail BI Report

Pivot table reports are available in the tools, such as Microsoft SQL Server 2005 and above. However, due to computing complexity, increased processor time and other time consuming factors, such tools are often cumbersome to use. A similar report developed in a BI system will only take a few seconds to show results. Here is a case study on how we built additional reports in SSRS for an ERP system.

Due to the limitations of ERP systems, senior management uses spreadsheets for analysis and presentation. There are some challenges on sharing these spreadsheets with other members in the organization such as:

1. There is no single version because a new sheet has to be created each time.
2. It is difficult and time consuming to manually prepare the sheet using data from multiple applications/systems.
3. There is a lack of standardization as different sheets are prepared by different team members.

These challenges are overcome by using a BI solution. Additionally, BI solutions using services, such as Microsoft SSRS, also provide custom report development functionalities, dashboards, data visualization, and drilldown capabilities.

BI vendors like Optimus provide report development as a managed service to the organizations with need for reports and dashboards on an ERP system. If you would like to learn more about our report development services, please feel free to contact us. We will be happy to sit down and discuss your needs.

Sketches are casual so stakeholders aren

Lately I’ve been using an excellent web/desktop application, Balsamiq, for creating interactive mockups. It can be used to create a screen-by-screen overview of a proposed mobile, desktop, or web application. Recently I mocked up a 30+ screen mobile/web client then exported it to PDF to share with a client.

In addition to exporting your designs to PDF, you can launch an interactive prototype via a web browser. This can be shared with anyone on the web or you can invite users to add comments or even collaborate.

Steps for Mocking up a Software Solution:

1. Establish Requirements: list the system’s business requirements.
2. Create high-level use cases: list the main actions a user will take. It’s best to write use-cases or tasks from start to finish. For example: Log into system, go to settings, update email, save, log out.
3. Create template: creating a basic template for your application will save on re-work. Every application generally has a framework of common elements. By creating a template you can ensure you have one standard look/feel/functionality throughout the application.
4. Create your mockups! The exciting part is putting all the pieces together. I advise going through the list of use-cases and creating them one at a time. By the end you should have all required screens setup in a very user friendly fashion. By focusing on use-cases instead of requirements, you’re forced to focus on user experience. Good software is intuitive and quick. Common tasks should be no more than a few clicks/taps away.
5. Run your prototype: once you have created a few screens and links between them, you can run your prototype in your browser. Although it won’t feel the same as an actual application, you start to get a really good idea of the system.
6. Share your mockups! Sharing is easy. You can either copy the prototype URL (which are public by nature but nearly impossible to guess) or export the PDFs. I recommend both.

If you’re looking to get some software developed, I strongly recommend playing around in Balsamiq and getting your ideas “on paper.” By putting your ideas into an interactive medium, it’ll be easier to imagine the final product.

QC is a quality management platform that offers requirements management, tests management, defects management, and business process testing; these tasks are handled by different modules in QC. I am going to highlight some of the most important features of the following modules:

• Business Components
• Test Plans
• Test Lab

Business Components

The Business Components module enables us to create and manage business components. The tabs in the module allow us to describe the component, attach a snapshot, edit the parameters, detail the design steps, and automate the component. The “Parameters”, “Design Steps”, and “Automation” tabs are used more often in automation.

These features are essential for automating and managing business components:

• Provide the component overview in “Details”
  Having tens of components for one project, sometimes you may forget or get confused about what each component does. The “Details” tab allows you to provide the component summary and other information, so it makes finding a component and understanding the purpose of a component easier.
• Become a workspace for keyword-driven components
  If the component is a scripted component, there is only a “Launch” button in “Automation” for you to launch QTP and develop the test script in QTP. On the other hand, if the component is a keyword-driven component, “Automation” becomes a workspace for creating and modifying the test steps. Test steps can be specified by using the object repository associated with the application area, choosing an operation (operation is a function associated with the application area, or a general operation that is provided by QTP, such as “OpenApp”), or adding a comment.
• Display detailed test steps in “Design Steps”
  When you want to know in details how a component executes a task in each test step, the overview in “Details” may not give you enough information. “Design steps” tab allows you to read and edit the test steps of the component, so you don’t have to turn to the test script itself for this information. If the component is a keyword-driven component, the design steps are filled automatically according to the test steps in “Automation”. However, if the component is a scripted component, the design steps are written manually.
• Display the dependencies to tests, resources, and application area
  The “Dependencies” shows you the test resources and the application area that the component is using, and the tests that deploy the component. If you want to make changes to one entity (resource, application area or component), you can analyze which entities are impacted.

Test Plan

The Test Plan module allows us to plan a test process by using the business components defined in the Business Components module. In fact, different types of tests can be created in this module, such as a WinRunner test or a QTP test. However, since the scope of this article is on BPTs, I will only mention how to handle these tests here.

The following are the steps to create a business process test:

1. Click the “New Test” icon and select “BUSINESS-PROCESS” as the test type
2. In the “Test Script” tab, clicking “Select Component” brings up a list of components in the Business Process modules for us to choose from. Choose the components required for the test and arrange the order.
3. Configure the input parameters by specifying the value or linking output parameters to the input parameters. An input parameter can be defined as a run-time parameter. A run-time parameter is defined when the test is run, so tester provides the parameter value at run-time. On the contrary, if a parameter is configured with a value, this value used for every test run, and tester does not have to provide a parameter value every time the test is run. To define a parameter to be a run-time parameter, simply type “{}” in the Parameter Value. For example, if “Username” is a run-time parameter, then set the Parameter Value as “{Username}”.

Test Lab

The Test Lab module lets us create test sets, schedule test runs, and view test results. This module has some useful features:

• Specify the run-time parameter values with test iterations: If a test has undefined parameter values (i.e. run-time parameters), QC enables us to assign different values to these parameters with the iteration runs. This feature is very helpful when you want to test different scenarios.
• Schedule test execution date and time: You can schedule the test to run at a specific date and time. QC will kick-start the execution for you.
• Assign the test machine to execute tests: Tester can assign a host (local machine or a remote machine) to run the tests. This option becomes very important when you want to run the tests in different systems and environment.
• View test results: The input and output parameters, error messages and test run statistics of previous test runs are displayed in Test Lab.

QTP and QC are two very strong tools for test automation and test management, and they offer a lot more functions than listed in my articles. I hope my articles will give you a good start on using them.

As the only Microsoft Gold Partner on the Data Platform in Western Canada, we have helped our clients develop reports faster, better, and cheaper.

Since companies use multiple systems for different business functions and each of these systems have a database, data needs to be gathered, combined, and compared. This provides an opportunity to gain valuable insights into the market, the customers, and the core business.

Here are five case studies describing the report development services we’ve provided in the last few months:

1. See how we developed reports for Microsoft Dynamics AX using Microsoft’s SSRS (Case Study).
2. See how we helped a client migrate reports from Crystal Reports to SSRS (Case Study).
3. See how we used SharePoint, PerformancePoint, and SSRS to create a web-based reporting portal (Case Study).
4. See how we helped a client extract more data from their system by building an external reporting solution (Case Study).
5. See how we partnered with a software development company to enhance their reporting offering (Case Study).

What is “Report Development as a Managed Service”?

Report development as a managed service means our clients have on-demand access to a diverse reporting team. Although we are a MS partner, we have extensive expertise on other proprietary and open source platforms.

As explained in my previous blog on “Benefits of Report Development as a Managed Service“ our clients receive the following benefits:

• Overhead cost control
• Agile report development process
• Access to experts on multiple platforms
• No risk of losing skills from staff turnover
• Continuous availability and on demand bandwidth
• Industry standard IT governance

Due to the advancement in technologies these reports, dashboards, and scorecards can be:

• Published to smart phones and tablets
• Published to emails as attachments
• Published to a web portal
• Published within a third party application
• Scheduled to run automatically
• Run on-demand

These days IT department in big organizations are seen as service centres. Whether they provide services in house or outsource to a vendor, IT departments charge back the expense to the divisions or departments for which the services are performed.

Business Intelligence vendors like Optimus provide managed services to provide benefits to these organizations. If you would like to learn more about our report development process and experience, please feel free to contact us. We will be happy to sit down and discuss your needs.