Data can be an engineering team's superpower if used correctly. But, let's admit it.
We've seen a lot of bad practices out there...
Let's talk about some dos and don'ts of using software metrics so we can rake in all the great benefits without introducing bad practices.
First, some high level rules:
Less is more
Keep it high level
Keep it team based
Comparisons are not fair
No metric lives alone
Let's dive in...
Rule #1: Less is more
A plan that focuses on too many metrics is doomed to fail. Nobody can focus on 50 metrics or goals. Ultimately this will demotivate the team. Focus on a few, key metrics so the team can focus and drive success.
Takeaway: Focus on key metrics (Change Lead Time and Throughput)
Rule #2: Use a North Star
Metrics should be high-level and provide a north star for the team. Getting a 10,000 ft. view helps us see the forest from the trees and get a better sense of where we can improve.
Takeaway: Focus on high-level trends & patterns.
Rule #3: Teams not Individuals
Software is a team sport. Let's act like it. Haystack is geared towards team improvement. We should be focused on what helps the overall team improves, not searching for who is falling behind.
Takeaway: Don't point out individual engineers.
Rule #4: Humans Required
Data is not a replacement for good old fashioned communication. Metrics are great indicators or red flags, but they never account for the full explanation. They help understand what’s happening but we still need humans to understand why it’s happening.
Takeaway: Always provide context with metrics.
Rule #5: Comparisons Are Not Fair
Teams, projects, and engineers should never be compared. They all have different skills, experiences, and dynamics. We should be looking to improve at all levels, not compare to one another. Comparisons hurt engineering culture.
Takeaway: Look at each team independently. Comparisons are not welcome.
Rule #6: Questions > Conclusions
"You need to do this faster" is something we all hate to hear. Metrics help signal when problems arise but when this happens, we should enable our teams to do what they do best - communicate, collaborate, and problem solve. Rather than coming in with conclusions, our metrics should help us ask the right questions. A better question would be: "Does this look like something we should address?"
Takeaway: Bring questions, not conclusions.