DevOps Quality is a Business Requirement - Engineering Insights Podcast Ep. 1

Download/Listen

anchor.fm/haystack-analytics

Transcript of main content (presented by Junade Ali)

I recently read Team Topologies by Matthew Skelton and Manuel Pais. Finding that technical architecture often follows organisational structure (through Conway's Law), the book describes patterns for organising technology teams to reduce cognitive load and accelerate the flow of change.

In doing so, the book describes essential best practices to avoid anti-patterns developing; from the importance of software architects needing to be managers too, to only building a Thinnest Viable Platform to avoid poor developer experience from excessively overcomplicated internal platforms.

The book builds on prior work by Nicole Forsgren, Jez Humble and Gene Kim. Through years of State of DevOps Reports and summarising their findings in the Accelerate book, the authors analysed 23,000 different profit & not-for-profit organizations of different sizes, at varying stages of digitisation.

The authors of Accelerate identified that, when measured against just 4 key metrics, the “highest performers are twice as likely to meet or exceed their organisational performance goals.” In other words, these indicators would lead to higher rates of profitability, market share and customer satisfaction for their respective companies.

The authors of Team Topologies have designed their organisation patterns around driving organisational improvements to these 4 key metrics. Whilst such organisational structure is important, in my view, it is not sufficient.

When an electronics engineer selects which resistor to use in a project, they can select between one with a gold band, a silver band or no coloured band at all. These coloured bands determine the error tolerance that is acceptable from when a resistor was manufactured. The selection of these components by an electronics engineer depends on the needs of the overall system, which is in turn informed by the product requirements.

Likewise, when we build internal DevOps Platforms, we must consider the requirements at hand. A DevOps platform for low-risk websites might be very tolerant of allowing higher a Change Failure Rate and a longer Mean Time To Recovery, at the benefit of having a far shorter Cycle Time for changes. The reverse may be true for systems that affect human life.

The ability to balance risk and reward is fundamental to engineering. How we assess this balance and communicate it is fundamentally based in the real world and how our systems are used.

Similarly, changes to an internal Platform that introduces new technology that degrades your ability to ship as quickly (i.e. a lower Cycle Time or reducing the Number of Deployments) must be considered carefully. As an Intercom blog post elucidates with its title: "Shipping is your company’s heartbeat". It is often tempting to add more technical complexity to a system, especially with new cloud computing services constantly appearing, but sometimes you need to focus on driving down the complexity so you can serve your business requirements well.

In the words of Woody Guthrie: “Any fool can make something complicated. It takes a genius to make it simple.” This is more crudely summarised in the KISS principle as: "keep it simple, stupid".

Whether product performance (measured metrics like product usage or satisfaction) or engineering performance (measured against Accelerate's Four Key Metrics), it is essential to bake the measurements you care about into your requirements and measure against them continuously. Like scientists increasingly preregister studies, so they can't later change their hypothesis to fit the data, it is important to ensure you focus on measuring what really matters to your organisation.

Robust quantitate engineering metrics (like Accelerate's Four Key Metrics) are vital to getting feedback on how your organisation is actually performing at DevOps. Indeed, my own belief in the power of quantifiable engineering metrics is so great, I decided to pivot my own career to helping creating an engineering metrics platform at Haystack Analytics.

To drive improvements to these metrics, DevOps Platform teams must their Platform as an internal product and focus their business goals around quantifiable DevOps quality metrics.

‍