The Impact of DevOps on Testing

A fundamental principle of DevOps is that the concept demands close collaboration between all the stakeholders involved in an end-to-end software production process from requirements to development to deployment and delivery.

However, because the portrayal of DevOps often exclusively emphasize melding development and IT Operations, other stakeholders may seem peripheral to the process. In particular, many outside observers assume that the continuous development, continuous integration and continuous delivery practices intrinsic to DevOps somehow reduce the need for testers.

In fact, DevOps testing is a primary contributor to assuring that the software delivered is of high quality and meets customers’ actual needs. It is also a fact that testing teams need to increase their contributions in the world of DevOps as well.

The Effect on DevOps without Integrated Testing

If an organization fails to recognize the crucial role QA plays within a successful DevOps transformation, it will be unable to achieve smooth operation of a continuous delivery model. Developers will continue to unit test with functional-only tunnel vision and hand over the remainder of testing to QA at their own pace. This situation creates an almost invisible disconnect between development and operations that creates undue friction toward reaching continuous delivery goals.

DevOps Testing Shifts Leftward …

A DevOps shop meeting its goals involves testers at all phases of the development process. The first step must be removing legacy boundaries between QA and development organizations in order that their processes merge and operate in parallel. Testers must integrate themselves tightly within development teams by working hand-in-hand as code is produced instead of waiting at the other end of an artificial pipeline. Dedicated test resources embedded in scrum meetings is a good, though insufficient, starting point.

A leftward shift in testing supports a DevOps philosophy of fail early, fail fast and fail often with respect to functional defects. It also enable verification of customer requirements at the head of the development cycle by the test team in collaboration with the production end of the CI/CD cycle.
To be completely effective, however, early testing responsibility must be shared by both testers and developers. QA’s role in this is to research and acquire the latest, most effective techniques, tools and technical expertise to make effective early testing processes happen with minimal disruption.

… and Rightward

Test teams in DevOps have a clear charter to increasingly use automation to streamline their own testing efforts. Additionally, test teams can play a logical role of integration between development and IT Operations to proactively increase system stability in the later phases of continuous delivery.

Beyond responsibility for automating builds and integrations, however, they should extend their automation skillsets to include automated tools for development infrastructure acquisition, configuration and provisioning. They are in the best position to evaluate time and resource requirements from both Dev and Ops, and how to serve these needs in the most efficient manner.

Furthermore, since their new role in DevOps extends beyond defect discovery into evaluating deployment/customer requirements, they can facilitate a leftward shift of Ops by adopting or facilitating other processes as well such as requirement management and production monitoring functions.

Continuous Testing Is Paramount

Regardless of how far left or right testing teams expand their responsibilities, their overarching obligation is creating a continuous testing approach that spans the entire CI/CD cycle. This means testers must be cognizant of early design and requirements decisions and provide feedback based on their expertise and experience, plus respond to later-phase testing and production requirements.

Early involvement enables tester to prepare their own requirements and implement test scripts even as the first lines of code are written. As development proceeds, involved, knowledgeable testers are key to ensuring that developers receive immediate feedback on defects and understand the impacts of their implementation decisions on later stages of a CI/CD pipeline, such as performance testing, infrastructure scaling and disaster recovery.

Clearly, to effectively complete a realistic, productive DevOps transition, testing plays an enormous role. In fact, its role expands and tester technology expertise, especially with respect to automation, significantly increases. Additionally, testing is uniquely positioned to further increase its value by stretching itself into areas traditionally allocated to Ops in a general leftward shift of shared responsibilities.

Far from being diminished, in a DevOps context testing becomes an ubiquitous function and tightly integrated into all the myriad “continuous” processes central to a successful DevOps conversion.

Some may argue that extending the testing organization’s reach both leftward into Development and rightward into Ops blurs the distinction of testing objectives and may lead to a dilution of its core value.

To view it as “blurring,” however, is a glass half-empty stance that overlooks key tenets of DevOps, which are reducing specialization, increasing collaboration and broader sharing of responsibility across artificial process boundaries. Without those principles in place and extended to test teams, the real benefits of DevOps are impossible to achieve.


1 reply
  1. Vipul Kulshrestha says:

    WRT “QA’s role in this is to research and acquire the latest, most effective techniques, tools and technical expertise to make effective early testing processes happen with minimal disruption,” proactive action is a key implied factor which impacts the outcome significantly. Thanks

Comments are closed.