What Is Exploratory Testing?
Exploratory testing may elicit connotations of hacking through a jungle at random, but in the context of software testing it is a disciplined investigative process with a well-defined role and guidelines that lead to an efficient and effective testing approach. In a nutshell, it is a simultaneous combination of learning, designing tests and executing those tests. The emphasis is on quick and useful results versus the development of formal test cases, test scripts and so on.
How Exploratory Testing Enhances Agile Development
Agile software methodologies are centred around brief bursts of development/test activity called sprints or scrums. Although each sprint’s stages of planning, estimation, coding, integration, testing and release are similar to those used in other methodologies, the timeframe is compressed to a few weeks.
For sprints to work well, it is critical that developers receive instant feedback on their work. Exploratory testing is particularly apt for this situation. It enables testers to rapidly familiarize themselves with the software features and behaviour, quickly design tests based on rough specifications and execute those tests on the fly to validate software functionality and uncover unplanned defects.
Exploratory Testing Complements Test Automation
The length of agile sprint cycles makes it difficult to squeeze in the overhead of formal test automation. Furthermore, most manual exploratory tests are run once or only a few times, which makes them incompatible for automation.
However, the development of automated unit test scripts is often possible within developer IDEs. These are collected into individual component test suites. These scripts may be added to the integration regression suite, which in the case of agile methodology may be run multiple times per day.
Exploratory testing spans both unit and integration levels of testing. Individual exploratory tests are evaluated from sprint to sprint and those deemed of high value are converted to automated test scripts. Additionally, the knowledge acquired during each cycle of exploratory testing guides the design of subsequent automation test suites.
Effective Exploratory Testing
The key to an effective use of exploratory testing is to bear in mind that it is an adaptive process rather than a predictive one. A balance must be struck between the freedom to explore and formal tracking methods aimed at pinning down metrics such as defect rates and code coverage.
The Ideal Exploratory Tester
Although exploratory testing often benefits from being performed by both technical and non-technical staff including clients, the best results come from using skilled testers with specific attributes:
- Test design skills
- Testing tools proficiency
- Non-linear thinking abilities
- Excellent observation skills
- An open and inquiring mind
- Excellent communication skills
Testers should be comfortable with working independently guided by sparse requirements.
Test charters/outlines are more useful than formal test cases when performing exploratory tests. They provide more degrees of freedom and less overhead. These may be guided by functional areas and developed from heuristics:
- Designing tests based on the “nouns and verbs” of what the software is accomplishing
- Identifying and testing all possible paths to obtaining equal results
- Creating tests suggested by the context of the system within which the software runs
- Generating tests based on all explicit, implicit, direct and indirect variations in the software’s state and transitions
- Developing boundary condition tests based on data, workflow and time
Within each test charter or via specific charters, exploratory testing should always identify high risk areas to assign extra attention during subsequent sprints.
To get the most out of exploratory test sessions, frequent stand-ups will gauge progress and facilitate adaptation to areas deemed to need more in-depth exploration. The most valuable tests should be integrated into automated test scripts.
Exploratory testing is one of many effective techniques in the pursuit of software quality. It is frequently misunderstood as an unguided, random process, which it is not. The proper application of exploratory testing significantly enhances the effectiveness of the entire testing effort. It is especially powerful within the tight development cycles typical of agile development methodologies.
When applied early in any software methodology, however, it produces rapid results that validate development direction and progress, gauge design risk and pave the way for subsequent creation of high-value automated testing.