Software engineering is an industry that is maturing. Due it the industry being so young, there are many experiments being done on tracking metrics. In this article, we will go over metrics that are commonly used as top-level metrics (or goals), rather than leading metrics to give information on the current bottleneck.

Goal: Improve the quality of the product.

Expectation: If we measure test coverage we should see an improvement in testing, thus improvement in the quality of our codebase.

Reality: In reality the number of poor tests increases. Engineers are doing test sprints, delaying features to write tests, and try to meet a minimum % that was picked. The test themselves becomes poor. We didn't say how to test better, we just said test more.

Recommendation: Focusing on the outcomes, rather than compliance to a number. The more we increase deploy frequency, the better we get at managing defect rate. Essentially, smaller batches of high-quality work deployed fast would allow the quality of the product.

We recommend measuring Deployment frequency and Defect Rate.

Goal: We want to be more predictable.

Expectation: If we look at WIP/completed work, then the team would be better at completing tasks in a timely manner.

Reality: The team focussed on planning instead of working. To meet the number that was set, the team would do very detailed planning and meet their goal. We get predictability at the cost of delivery speed.

Recommendation: Focusing on the outcomes, rather than compliance to a number. Focus on improving the flow of delivery. Create smaller batches that can be more consistently delivered.

We recommend measuring Change Lead Time and Defect Rate.