Blogs

Why Past Attempts of Software Metrics Have Failed Us

Julian Colina
September 7, 2020

TLDR; Software engineering is a black box. It's incredibly complex and nuanced. The industry makes attempts to stack rank and measure via simplified metrics. This is the wrong approach and hurts engineering culture. Software is complex, previous attempts don't take that into account. There is a better solution.

Software engineering is a black box because

  1. Software engineering is complex and invisible
  2. The output of software development varies

There have been several attempts at quantifying developer productivity. The limited content you'll find fits into a few categories.

  1. KPIs used to measure software engineer performance
  2. Methods of measuring developer performance and their fall backs
  3. Opinion pieces on if measurement is even possible

You'll notice a pattern prevalent in the space. The industry is obsessed with finding

  1. Golden metrics to stack rank and compare teams and individuals.
  2. KPIs to objectively determine an engineer's performance.

The result?

Attempts to measure software engineer and team productivity using the same, simplified metrics and KPIs.

But that's the wrong approach..

  1. Attempting to stack rank is fundamentally flawed
  2. 'Successful outcomes' are situational
  3. Simplified metrics hurt engineering culture

Focus on productivity, not yard sticks

  1. Measure engineers and teams independently
  2. Compare relative to individual history
  3. Identify the main risks and bottlenecks of productivity

Why is this better?

  1. Accurately measures productivity
  2. Handles situational outcomes
  3. Supports engineering culture
  4. Enables early risk identification

Main takeaway:

From the nature of the work to the culture that supports it, engineering is incredibly complex and nuanced. Previous attempts at measuring engineering performance have done a poor job of taking this complexity into account. Making attempts to measure each engineer and team against the same yard stick not only hurts culture, but gives no true indication of productivity to begin with.

To date, the industry has been attempting this 'holy grail' of software development metrics from the wrong perspective. By focusing on drivers and blockers of productivity rather than KPIs and stack ranking; we can start to measure engineers and teams in a healthy way. More importantly, this new perspective enables engineering leaders to introduce a software development metrics to help improve while maintaining the highest impact driver of productivity (culture).

Check out Haystack's approach here!

Haystack helps engineering leaders identify blockers and trends. Directly from Github. Instead of guessing if you're improving, or constantly bothering your team for progress updates, simply use Haystack to get alerts in your inbox every morning. Plus a dashboard to track improvements over time.

Try it for free

Julian Colina
CEO & Cofounder of Haystack.

Our latest news

Discover our latest articles, feature releases, and more!

Ready to get started?

Start Free Trial