Haystack vs. Waydev

More Harm Than Good?

Waydev market the fact that their service can be used for comparing developers, offering multiple tools for doing so. They also offer features like "inactivity reports" to aid in micromanagement practices. Evidence suggests these are harmful practices which don't aid team performance.

In Project Oxygen, Google evaluated manager performance ratings and manager feedback from Google’s annual employee survey - they found that great managers are excellent coaches, but they do not micromanage.

Google also studied team performance in Project Aristotle evaluating a total of 180 teams (including 115 project teams in engineering) using hundreds of double-blind interviews and quantitate data sources. Google's researchers found that individual performance of team members is not significantly connected with team effectiveness, but psychological safety is essential.

At Haystack Analytics we firmly stand against the notion that individual level metrics can provide actionable insights and believe that they cause more damage to work culture as they are invasive, stress-inducing and misleading. As such, we only track delivery at a team level delivery. We also provide additional insights to aid managers in preventing risks like burnout.

Instead, Haystack Analytics Integrates with Slack and email to provide real time comprehensive alerts around sprint risks, blockers and burnout. This means that teams can collectively address risks like blocked Pull Requests, code being merged without review or burnout, without needing to adopt harmful and authoritarian approaches. Managers can also gain visibility into these risk factors.

Waydev does not provide alerts but gives a daily summary of work done. This means manager frustration can mount by not getting insight into risk factors early enough, ultimately harming both engineer and manager well-being.

Waydev market the fact that their tool can be used provide metrics for individual engineers in one-to-ones, at Haystack we believe engineering management is a skill. Whilst alerts for risk factors can support management, simply reading out harmful "developer productivity" metrics during a 1:1 with your team is likely to do more harm than good. Haystack does not offer this kind of product functionality and we encourage customers seeking such functionality to find an alternative vendor.

No Snake Oil

At Haystack Analytics, we only provide software engineering metrics that are proven to help teams - either through our own validation or empirical scientific evaluation.

For example; our North Star metrics include the Four Key Metrics. Rigorous appraisal of these Four Key Metrics (in research led by Dr Nicole Forsgren) has shown that higher performers are 2x more likely to meet their commercial goals (productivity, profitability, market share, number of customers) and their non-commercial goals (quantity of products or services, operating efficiency, customer satisfaction, quality of products or services and achieving organisational or mission goals). Indeed, companies which do well under these DevOps metrics have a 50% higher market cap growth over 3 years. The mechanics of how metrics drive organisational performance is also well understood and has been rationalised by industry leaders like Martin Fowler.

Unfortunately, many of our competitors don't apply the same level of rigour to validating their metrics. Indeed, in many cases unvalidated metrics may prove harmful and actively damage your organisation.

Waydev instead focuses on output metrics on an individual level providing executives with insights like Number of Commits and Number of PRs. Their dashboard is designed primarily to give a bird's eye view of your team's input, output and work habits instead of delivery insights. This makes it hard to get actionable insights.

Both Pluralsight Flow (previously known as GitPrime) and Waydev contain a code quality metric called "churn", this is defined as "code which is deleted or rewritten shortly after being written (less than 3 weeks old)". This metric is then used to compute an "efficiency" metric which is assumed to be "the percentage of all contributed code which is productive work" for a given engineer. Not only have we seen no evidence that such a metric will actually help your organisation, it appears to fly in the face of the evidence we have on the importance of lowering Cycle Time.

Companies with lower Cycle Time (time from development to code in production) are able to rapidly test ideas to gain quick customer feedback, resulting in product that is more likely to satisfy a customer. If you test something and your customers don't like it - you should be able to remove it without being punished for increasing some meaningless "churn" metric.

Support

We have a dedicated support team of experienced engineering leaders available on-demand from our Slack Community. Every Haystack user has a dedicated private slack channel with our team of experts to help you deep dive into your data and teach you how to improve delivery.

Waydev provides a dedicated manager only on their premium and enterprise tiers.

Security

Haystack is the only tool that doesn’t read, store, or persist source code. We use the Github API to pull metadata such as timestamps and commit messages to produce all our insights. We also maintain industry best practices to ensure that all user information is safe.

Cost

Haystack is charged on a monthly basis with each contract renewed at the end of the month, with our growth plan priced at $20 per developer a month when purchased annually. Monthly billing is available on our start-up plan at $25 per developer a month.

Waydev requires that contracts are locked in for an annual rate of $449 per developer a year (~$37.42 per month) for their entry level pricing.