The evolution of software testing services.

I’m not sure who coined the word ‘Software-as-a-Service”, but Salesforce.com have certainly capitalized on it and packaged it as a concept, an idea, a way of thinking and most importantly have successfully monetized it.

Any on-demand service has an extension of ‘as-a-service’. Testing is no exception. In the following few paragraphs, I will try to map this concept with real world scenarios and hopefully give our readers a flavour as to where we are heading, from a Testing & QA perspective, in the next few months to come.

Over the past couple of months, we spoke to over 50 companies in British Columbia, Canada. This included the likes of CIOs, VP Engineering, IT Directors, QA Managers and Testers.

We approached them with couple of pointed question:

  1. What are your pain areas around Testing & QA?
  2. What are your % budget allocation (quantitative) / mindshare (qualitative) to Testing & QA?

 

Our take away from all the meetings:

  1. 85% of the target audience were certain that they need help on Testing & QA.

Of this 85%, they can be categorized into 3 categories:

  1. Needing help on improving their overall QA processes.
  2. Needing help on plain execution of their test cases.
  3. Needing help to move towards higher level of automation.

Incidentally, although Testing & QA is an area of importance, decision makers are unable to quantify the budget allocation nor able to dedicate mindshare to it. The primary reason is the apprehension of needing on-going testing support. They would rather prefer it on an on-demand basis.

This was enough food for thought for us to realise the benefit of ‘Testing-as-a-Service’ and how it can help organisations lower their total cost of ownership and guarantee predictable outcomes, across the lifespan of the development cycle.

Testing has come a long way from being a FTE (full-time equivalent) driven way of working, to ‘as-a-service way’ of thinking.

The evolution of software testing services.

*Source: Optimus Information

We all remember those good old days of multi-year contracts, where vendors use to layout the buffet and testing use to be just another line item. Then the offshoring wave kicked-in and dedicated testing teams were deployed overnight. The concept of the Testing-Center-of-Excellence started taking shape. This resulted in segregation in Development and Maintenance contracts which led to the foundation of Independent Testing Vendors across the globe.

However current market shifts compelled organisations to take a step back and rethink their business model. Incidentally, Testing & QA fell into the on-demand category. Testing-as-a-Service thus became an evolutionary approach, where the risks are further being distributed across the board.

As we delved further into it, we observed some key subtleties of this concept:

  1. Testing-as-a-Service is not the same as cloud based testing. The latter is an activity / means to an end, while Testing-as-a-Service is a framework, a unique way of approaching Testing & QA, independent of vendors and tools, which traditionally been considered as an activity.
  2. Defining the objective of the Testing-as-a-Service model: 1) Predictable outcomes, 2) Move towards an on-demand pricing model 3) Lower TCO. Organisations should demand a result based invoices rather than effort based invoices, raised typically at the end of every month. This will be factored into the model.
  3. Start with areas where possibility of automation is greater. Testing-as-a-Service model works very well where need is well defined and understood. This might seem like a ‘catch-22’ situation for certain organisations. In this case, this model may not be the best one to begin with; however necessary due diligence should be done for future benefits and adaptation of this model.
  4. Organisations should work closely with their testing tool vendors to factor in licensing cost in the model. One of the approaches is to amortize the license cost of the tool over the span of the usage. However, leading vendors such as HP, SmartBear, Microsoft, offers an on-demand licensing portfolio.

All said and done, these are some industry accepted approaches. In the end, focus should be on customer needs and to ensure a win-win situation for all stakeholders.

These are some of our thoughts as we continue to work closely with our clients as their trusted advisor. Any pointed feedback is always welcome; as it would help us further improve our processes.

*This article was also featured in ‘SoftwareTest Professional’ Magazine’s January Newsletter, a leading industry magazine focused towards Testing Professionals. Seth Eliot, a thought-leader in the testing community and a Senior Knowledge Engineer, Test Excellence, at Microsoft, took personal interest in the article as well.