Stop starting. Start finishing.
David J. Anderson.
Knowing your deployment frequency helps understand the rate at which you're delivering value to customers. With the best teams deploying on a daily basis, this is a key North Star Metric to help keep your team on track and delivering.
Haystack can calculate deployments in 5 different ways
Commit that has been tagged (Git tags)
Commit that has been tagged that contains a certain string (Git tags contains)
Pull Requests merged into a specific branch (e.g. merged to master or main)
Pull Requests containing a title (e.g. 'release-v*')
Pull Requests containing a label (e.g. 'deployment')
All of the different methods essentially work the same. Haystack creates an internal "Deployment" entity whenever there is an item that matches the rule. We calculate the deployment time based on the commit date for git tags and pull request merge date for the rest.
Learn more on how to configure deployments.
According to DORA;
Change Failure Rate, measure what percentage of changes to production or released to users result in degraded service (e.g.:, lead to service impairment or service outage) and subsequently require remediation (e.g., require a hotfix, rollback, fix forward, patch)?
Haystack calculates service deployments that resulted in service impairment by tracking hotfixes. There are 3 different ways this can be done.
Pull Requests containing a title (e.g. 'hotfix-*')
Pull Requests containing a label (e.g. 'hotfix', 'rollback')
Pull Requests containing branch (e.g. 'hotfix/', 'fix/')
Both of the calculations work the same way by pattern matching and tagging the pull request as "Hotfix". We take all of the "Hotfix Pull Request" and map them in chronological order with deployments. If there is at least one "Hotfix Pull Request" included inside a deployment, we mark that deployment as "Hotfix Deployment".
Change Failure Rate will be calculated as Hotfix Deployment / All Deployments.
Let's go over 3 scenarios:
Scenario 1:
In this scenario, the deployment and failure rate settings have been configured as follows.
Deployment Settings: Commit that has been tagged (Git Tags)
Failure Rate Settings: Pull Requests containing a label (e.g. 'hotfix')
Let's say we have 4 pull requests and 2 commits tagged in a time period.
The first pull request was merged.
A new commit with the tag "0.1.2" was committed and tagged as deployment.
The second pull request was merged and has a label including "hotfix", thus it was tagged as a hotfix pull request.
The third pull request was merged.
The fourth pull request was merged and has a label including "hotfix", thus it was tagged as a hotfix pull request.
A new commit with the tag "0.1.3" was committed and tagged as deployment. Since the deployment includes 3 pull requests where 2 of them are tagged as "Hotfix Pull Request", we tag the deployment as "Hotfix Deployment".
Our deployment frequency in this will be 2 deployments and the change failure rate will be 1 / 2 = 50%.
Boxes represent deployments, orange color represents hotfixes
Scenario 2:
In this scenario, the deployment and failure rate settings have been configured as follows (same as scenario 2)
Deployment Settings: Pull Requests merged into a specific branch (e.g. merged to main)
Failure Rate Settings: Pull Requests containing a label (e.g. 'hotfix')
Let's say we have 4 pull requests in a time period.
The first pull request was merged into "main" branch and tagged as deployment.
The second pull request was has a label including "hotfix" and was tagged as a hotfix pull request (note: this pr is not merged into "main" branch)
The third pull request was merged into "main" branch and tagged as deployment.
The fourth pull request was merged into "main" branch and tagged as deployment.
Since the hotfix pull request was not a deployment pull request, we mark the deployment where this pull request is released as hotfix deployment (represented in orange). Our deployment frequency in this will be 3 deployments and the change failure rate will be 1 / 3 = 33%.
Boxes represent deployments, orange color represents hotfixes
Scenario 3:
In this scenario, the deployment and failure rate settings have been configured as follows.
Deployment Settings: Pull Requests merged into a specific branch (e.g. merged to main)
Failure Rate Settings: Pull Requests containing a label (e.g. 'hotfix')
Let's say we have 3 pull requests in a time period.
The first pull request was merged into "main" branch and tagged as deployment.
The second pull request was merged into "main" branch and has a label including "hotfix", thus it was tagged as both hotfix and deployment.
The third pull request was merged into "main" branch and tagged as deployment.
Our deployment frequency in this will be 3 deployments and the change failure rate will be 1 / 3 = 33%.
Boxes represent deployments, orange color represents hotfixes
If none of the options above fits your deployment or hotfix process, let us know about your process so that we can add them. You can contact us from the chatbot or [email protected]
Check the following blogs: