What to Include in a Software Testing Traceability Matrix

A Software Testing traceability matrix is an essential tool for any thorough software tester. It should be referenced throughout the entire software development life cycle to bring transparency and completeness to software testing efforts.

Software Testing Requirements for Traceability Matrix

In simple words, a testing requirements traceability matrix is a document that traces and maps user requirements, usually requirement IDs from a requirement specification document, with the test case IDs. The purpose of this document is to make sure that all the requirements are covered in test cases so that nothing is missed.

The traceability matrix document is prepared to show clients that the coverage is complete. It usually includes the following columns: requirement, baseline document reference number, test case/condition and defect/bug ID. Using this document the person can track the Requirement based on the Defect id.

Adding a few more columns to the traceability matrix gives you a good test case coverage checklist.

Types of Traceability Matrices

  1. Forward Traceability: Mapping of requirements to test cases.
  2. Backward Traceability: Mapping of test cases to requirements.
  3. Bi-Directional Traceability:  A good example of a bi-directional traceability matrix used in software testing is the references from test cases to basis documentation and vice versa.

A forward traceability helps us see which requirements are covered in which test cases? Or whether a requirements is covered at all.

A forward traceability matrix ensures that we are building the right product.

A backward traceability matrix helps us see which test cases are mapped against which requirements.

This will help us in identify if there are test cases that do not trace to any coverage item. If the test case does not trace to a coverage item, it is either not required and should be removed, or else a specification like a requirement or two should be added. Therefore, this backward traceability is also very helpful if you want to identify that a particular test case is covering how many requirements.

A backward traceability matrix ensures that we are building the product right.

A bi-directional traceability matrix contains both forward and backward traceability.

Why use software testing traceability matrices?

The traceability matrices are the answer to the following questions when testing any software project:

  • How is it feasible to ensure, for each phase of the SDLC, that I have correctly accounted for all the customer’s needs?
  • How can I certify that the final software product meets the customer’s needs? It lets us make sure requirements are captured in test cases.

Disadvantages of not using traceability matrices include the following:

  • More defects in production poor or unknown test coverage.
  • Discovering bugs later in the development cycle resulting in more expensive fixes.
  • Difficulties planning and tracking projects.
  • Misunderstandings between different teams over project dependencies, delays, etc…

Benefits of using traceability matrices include the following:

  • Making it obvious to the client that the software is being developed as required.
  • Ensuring that all requirements are part of the test cases.
  • Ensuring that developers are not creating features that no one has requested.
  • Making it easy to identify missing functionalities.
  • Making it easy to find out which test cases need updating if there are change requests.

How to create a traceability matrix

  1. Open Excel to create Traceability Matrix:
  2. Define following columns:
    1. Use case ID / requirement ID.
    2. Use case / requirement description.
    3. One column for each test case.
  3. Identify all the testable requirements in granular level from requirement document. Typical requirements you need to capture are as follows:
    1. Used cases (all the flows are captured)
    2. Error Messages
    3. Business rules
    4. Functional rules
    5. Software requirement specifications
    6. Functional requirement specifications
  4. Identity all the test scenarios and test flows.
  5. Map Requirement IDs to the test cases. Assume (as per below table), Test case “TC 001” is one flow or scenario.  SR-1.1 and SR-1.2 are covered .
  6. Now from below table you can easily identify which test cases cover which requirements and which test cases need to update if there are any change requests.
Requirement IDRequirement DescriptionsTC 001TC 002TC 003
SR-1.1User should be able to do this. x
SR-1.2User should be able to do that. x
SR-1.3On clicking this, the following message should appear. x
SR-1.4 x
SR-1.5 x x
SR-1.6 x
SR-1.7 x

This is a very basic traceability matrix format. You can add more columns and make it more effective. Here are some columns you should consider adding:

  • ID
  • Assoc ID
  • Technical Assumptions
  • Customer Needs
  • Functional Requirement
  • Status
  • Architectural/Design Document
  • Technical Specification
  • System Component
  • Software Module
  • Test Case Number
  • Tested In
  • Implemented In
  • Verification
  • Additional Comments

Here is another simple forward traceability matrix that we used on a recent project in Excel format.

 

Note: This blog post has been updated with new information.

Testing Infrastructure in the Cloud: Is it Right for Your Business?

cloud-testing Testing Infrastructure in the Cloud: Is it Right for Your Business?

In last six months, many of our clients have started asking about leveraging cloud-based testing infrastructure. We were intrigued by this development and have now used cloud infrastructure on a few testing projects.

Some key benefits of this approach include :

  • Pay as you go: Clients need testing infrastructure only for limited duration of the project. Cloud pricing helps there.
  • Scalability: Clients need certain infrastructure scale to test non functional aspects like scalability, availability etc. Cloud infrastructure’s scalability fits this need perfectly.
  • Capital vs. operating expense: With cloud infrastructure, it is easier to expense to a specific initiative. It brings more accountability.
  • Cost: Overall, our experience show that cloud testing tends to be cost-efficient. Part of that efficiency comes from lower overhead and reduction in IT staff costs.

So with all of these advantages, why is not everyone adopting cloud testing?

First, I would say that the trend towards cloud testing, and cloud infrastructure in general, is definitely strong and growing. Clients looking to replace infrastructure are considering the cloud as one of the top 2 options.

However, there are aspects like lack of knowledge, fear of uncertainty and issues related to organizational dynamics holding many companies back from embracing the cloud.

Despite these holdups, the case for cloud infrastructure is quite good and we expect to see much higher traction in next few quarters.

Is the cloud ready for testing infrastructure?

Our view is that cloud is not be ready for 100 percent of testing, particularly in areas where you need to capture some network issues in a real environment. However, with the launch of new tools almost on a weekly basis as well as the cloud’s suitability to simulate scenarios, cloud testing is rapidly maturing.

So, to summarize, if you are considering either setting up testing infrastructure or replacing your current infrastructure, we encourage you to evaluate some of the cloud-based options. To learn more on this you can contact us here.

 

Note: This article has been updated with new information.

Scrum Tools Comparison

Scrum Tools Comparison for JIRA, OnTime, Redmine

As a QA Professional, I’m always looking to be in the know about tools and techniques companies use to deliver software.  I’ve worked in companies that have used Waterfall, Agile, and a mix and match of everything in between. With that, I’ve been exposed to a variety of tools to track testing and Scrum project progress. But, often people ask me what is the best scrum tool? To answer this question I have created a scrum tools comparison between JIRA, OnTime and Redmine.

It’s almost a given that everyone has a grip on the tools they’re using.  However, no matter what company and tool, sometimes the tools just don’t quite fit what we are looking for.  Below are three tools that I’ve worked with and a quick highlight of the pros and cons of each.

 

 

 

JIRA:

So far, the tool I’ve used the most is JIRA.  JIRA tracks bugs, tasks, and other issues and comes with a set of reports for upper management.  JIRA is especially useful for tracking medium to complex projects, but may be overkill for organizations that handle smaller projects.  The Greenhopper add-on is especially useful for Agile Project Management.

Pros: The drag and drop functionality for tickets is handy, easy to pull stats from for reporting, very visual interface.

Cons: It can be quite costly to use JIRA depending on the number of users. All web-based; therefore, access to information is dependent on the internet (or internal connectivity).

on-time Scrum Tools ComparisonOnTime:

Axosoft’s OnTime is great for organizations where client feedback drives the product backlog.  Because OnTime is so customizable, it’s best for the organization to have their internal SDLC process in place before trying to implement this tool; otherwise, it can get hairy trying to put the pieces together while configuring OnTime.

Pros: Client issues can easily be ported into defect tickets, can define work flows to move tickets from different departments, intuitive to use.

Cons: Unable to add inline attachments to tickets, web interface has less functionality than the downloadable client.

Redmine Scrum Tools ComparisonRedmine:

Organizations that have plenty of different projects on the go can use Redmine to track issues, tickets, and time spent on each task.  Redmine can also act as your company’s central repository for documents and other project artifacts.

Pros: being able to move tickets from project to project, facilitates team collaboration and easily searchable.

Cons: UI is not as intuitive as other tools, needs more built in reports and dashboards.

What tools have you used to track projects and what do you like/not like about it?

 

Note: This blog has been updated with new information.