The premise behind context-driven testing is that software should be viewed first and foremost as providing solutions to real problems. In order to effectively test software, the problems it solves, the end-user needs and usage contexts must be taken into consideration when creating a test plan and applying tests. It recognizes that test processes must adapt to the testing environment itself, which includes the methodologies, skills and resources available within a particular test organization.
Testing if the Problem Is Solved
Functional testing ensures that each feature works as stated in the requirements. However, that only partially addresses how successfully the features taken together provide a complete answer to customer needs. Context-driven testing also takes those needs into account when designing test cases.
For example, say a doctor’s office acquires an application that assists the doctor in making a correct diagnosis. It may have a high accuracy rate and every button and widget in the application works perfectly. If it offers a diagnosis in a non-transparent way, however, the doctor may lose confidence in its diagnostic abilities, which leads to disuse. If the app fails to interface with other office programs, its usefulness is also diminished.
Furthermore, pre-release testing must be extremely rigorous in order to avoid the possibility of a lawsuit for an incorrect diagnosis. These are usage, environmental and perception contexts that must be considered thoroughly when designing the test plan.
Context-Driven Testing Principles
Beyond the factors considered above with regard to software use and its operating environment, “true believers” of the context-driven school of thought consider other, more subjective factors as essential characteristics of context-driven testing. This is evident in the commonly accepted principles of context-driven testing paraphrased here:
- The end product must solve the intended problem in the intended context. If it does not, then it does not work.
- The value of any “best” practice depends on the context in which it is applied. The phrase “best practice” should never be considered an absolute statement.
- he application of judgment and skill applied in a cooperative manner is essential to product testing success.
- The people cooperating in the testing activity are the most important aspect of a project’s context. This especially applies to the project’s stakeholders.
- Projects naturally evolve over time in unpredictable ways and thus, flexibility is critical.
- Software testing, when performed correctly, is a challenging intellectual process.
Several of these principles do not apply directly to the context of software in the field, but to the context of the testing environment including the methodologies, standards, so-called best practices, abilities of the testers, test tool capabilities and stakeholder expectations with regard to what, when and how to test the software under development.
The Practices of Context-Driven Testing
To follow context-driven testing precepts, apply a holistic approach that wraps up testing constraints and goals into processes and practices most appropriate for the problem being solved by the software within its testing context.
Flexibility is a key attribute in context-driven testing, which leads many to believe that agile methodologies and context-driven testing significantly overlap in how they operate and get results, which is true to some extent.
Regardless of the methodologies used by your organization, however, context-driven approaches hold value. In practice, they lead to several useful practices that enhance any testing process:
- Sharing information and asking lots of questions significantly improves consensus on what precisely is the context of the project from various viewpoints. It increases learning and transparency as well.
- The test plan should be reviewed by as many stakeholders as possible supplemented by experts in the application or testing domain who are not directly tied to the project. Obviously, this should be done as early as possible in order to minimize the impact of changes.
- The test plan should be considered adjustable as the project moves ahead because knowledge of the entire project/product context is unlikely to be known a priori regardless of how many questions were asked.
- Carefully consider the appropriateness of any test tool, testing practice or paradigm, especially if it is being used out of convenience versus applicability to the test problems at hand.
- Defer the decision of when testing is complete to the project stakeholders. This increases their attention to the project and allows the testers to concentrate on their job without making judgments about product readiness.
In the end, context-driven testing boils down to doing the best job possible for product stakeholders, especially end-users, with the resources you have available. By definition, it is an approach that highly values problem solving, collaboration, intelligent testing and the most efficient use of testers’ skills.