In this article, we'll share some tips and best practices to successfully get buy-in from your team. At Haystack, we've helped countless companies adopt delivery metrics and have seen the good, bad, and ugly.
What Evidence Is There That Engineering North Star Metrics Provide Better Business Outcomes?
The following video provides an overview of some of the evidence that lower Change Lead Time leads to better organisational performance:
Transcribed text of the quotes from that video can be found at the bottom of this article in the section entitled Quotes on Using Metrics.
First Step: Getting Buy-In
The first hurdle is always getting buy-in from our team. Getting team buy-in is dependent on how well they understand the purpose of measuring - and how much they trust you.
Unfortunately, metrics like Lines of Code have been used in the past to judge individuals and erode trust so it's normal to be met with a bit of skepticism.
Don't let that deter you. We've come a long way from the days of lines of code and with clear intentions and a solid plan - we can introduce metrics and strengthen team trust and culture.
To successfully convince your team, you must have:
An Obvious Purpose For The Metrics
Solid Reasoning For The Outcomes You Seek
Be prepared to answer the following questions:
Why do we need metrics?
What are we going to measure?
Who will have access to these metrics?
Why did we pick these metrics, and which others could we have chosen?
What is our plan to move these metrics forward?
How will these metrics impact individuals?
The primary thing to keep in mind is trust.
Your team should understand your use cases and intent. They should trust they will not be judged by these metrics or evaluated on an individual basis in any way. Metrics should be used to empower the team, make the change, and ultimately improve the developer experience.
Data should always be used to help the team, not hurt it. This should be made completely and utterly clear to each and every member of the team.
You don't need to know all the answers just yet. It's okay if you don't.
For metrics to be successful, you just need the team to trust your intentions and embrace the strategy presented to them.
Need Some Inspiration?
If you need some inspiration on how to present your use cases and intent to your team, then here is an excellent example.
Juan Pablo Buriticá (VP of Engineering, Splice) wrote a proposal for his team that outlined his thinking in the form of an RFC. From there, his team helped iron out the details that weren't clear and make it more robust through their questions.
Quotes on Using Metrics
Martin Fowler on Accelerate's Metrics
From his talk "What Does Tech Excellence Look Like?":
"These are really quite interesting because they boil down to a very important measure that many people like me have been talking about for a while which is that of Change Lead Time. Change Lead Time is, what is the period of time between somebody in the business world, or in the domain that the software supporting, having an idea of something they would like to see in software - and how long does it take from that idea to form and then go into software running in production."
"The faster you can cycle ideas into production, that gives you an advantage in experimenting with things and being able to try things out. And what the study indicated is that Change Lead Time had a direct correlation with how good overall an organization's performance was."
"The other factor that they brought up this Mean Time to Recovery and that's actually quite interesting in itself. There's a phrase that goes out there, and comes from John Allspaw, that Mean Time to Recovery is more important than Mean Time Between Failures for most failures. The point being that, by putting stuff into production it's more important that you can realize that something's gone wrong and fix it than it is to prevent the failures in the first place."
"That's not something that many organizations really understand, they put a lot of effort into trying to prevent things going wrong but one of the things that we're learning is it it's more important to detect and fix failures than it is to prevent that. At least the most of the time, at least the non catastrophic cases. "So that's an important part of the picture is being able to recover fast, but let's go back onto this Change Lead Time. Change Lead Time as I've said is a very key measure and the reason I said that is important is it because it allows us to get early information about whether our ideas are worth pursuing or not."
Nicole Forsgren On How DevOps Metrics Correlate With Organisational Performance
From her talk, "The Key to High Performance: What the Data Says":
"High performers, year over year, are twice as likely to exceed their organisational goals. Profitability, productivity and market share. [...] We added additional measures like non-commercial goals, and the crazy thing is we find this same 2x multiplier. High performing organisations are twice as likely to exceed their non-commercial goals; effectiveness, efficiency, customer satisfaction. By the way, the measures for these are drawn from the academic literature - highly, highly rigorous measures. For bonus points, in the 2014 study, we had a large enough dataset that I could actually pull from stock market data - and we find that high performers had a 50% higher market cap growth over the previous 3 years."