Despite its popularity, DevOps continues to remain an abstract concept, open to varying interpretations with most agreeing that it is about efficiency. To some it involves effecting cultural change, some define it in terms of incorporation of automation tools, while others view it as a merger of development and operations teams to work as a cohesive unit.
In this edition, we summarize 4 key sets of metrics that management consulting firm Accenture promotes for measuring success and tracking progress. We have found these metrics to be valuable in implementing DevOps best practices and have reproduced them here with permission.
DevOps Metrics Categories
Velocity:
Velocity metrics collectively tend to quantify the amount of work accomplished and the time taken to accomplish the tasks completed in-between release cycles. The metrics in this category include: a) Build duration, b) QA cycle time, c) Deployment duration, d) Deployment frequency, e) Number of new lines of code, f) Merge frequency, etc.
Quality:
As the name implies the metrics paint a picture of the overall code quality in terms of defects found by the Q/A team. These metrics include: a) Number of bugs and vulnerabilities, b) Test coverage percent, c) Test pass rate percent, d) Defect density, e) Defect reintroduction rate, e) Defect ageing, etc.
Stability:
These metrics allow one to develop an understanding of how reliable, repeatable, and consistent the processes are whether automated or manual. These measures include: a) Build pass rate, b) Deployment pass rate, c) Number of incidents in deployments, d) Mean time to recover, e) Change success rate, f) Number of hot fixes, g) Deployment downtime in hours, h) Uptime percent , etc.
Release Readiness:
These metrics collectively allow project managers and other executives to determine how close they are to releasing code to production. These include metrics like: a) Release confidence percent, b) Release cycle time, c) Release frequency, d) Release days remaining, d) Release percent completed, e) Release size, f) End-to-end traceability percent, etc.
______________________
There are several SCM, orchestration, automation, and other tools, like Jenkins, Selenium JMeter, Jira, Assembla, etc., that can aid in the measurement and collection of the above metrics. Additionally, once collected, they can be published as dashboards and analyzed using Splunk, Athena, BI tools, or simply Excel.
While it is difficult to provide details of the above metrics in a newsletter, readers are encouraged to research the above metrics on their own or reach out to Data Dynamics team for more information on these metrics and how to capture them. Readers may also choose to view Accenture’s blog which has served as an inspiration for this newsletter.
