Every engineering team measures something. Commit counts, story points completed, pull requests merged, sprint velocity, lines of code written. These numbers feel concrete and trackable, and they create a satisfying narrative of activity.
The problem: none of them have a reliable relationship with business outcomes. A team that ships 40 story points per sprint of the wrong features is not outperforming a team that ships 20 story points of the right ones. A team with a high commit rate that ships 15 broken builds to production is not outperforming a team with half the commits and reliable deployments.
Metric systems that optimize for activity instead of outcomes produce rational individual behavior that leads to poor collective results. Engineers learn to estimate conservatively so velocity looks good. Tech debt gets hidden in "sprint" scope. Features get shipped in states that don't actually solve the user problem. These aren't character failures — they're predictable responses to misaligned measurement.
This post is about building a metrics framework that tells you what you actually need to know.
The Problem With Velocity
Velocity — the average story points completed per sprint — is seductive because it looks like a productivity measure and can be tracked in a spreadsheet. It's also nearly useless for management decision-making.
The fundamental problem: story point estimates are relative to each other within a team, not comparable across teams or time periods. A team that consistently over-estimates and then "hits their numbers" looks high-performing. A team that estimates accurately and occasionally misses due to genuine complexity looks inconsistent.
More critically, velocity measures work completed, not value delivered. Completing 45 story points of features users don't adopt is not a better outcome than completing 25 story points of a single feature that drives meaningful retention. Velocity provides no signal about which of these you're doing.
This doesn't mean you should stop estimating. Estimation is valuable for planning. But velocity should not be your primary management lever for engineering performance.
The DORA Metrics: A Better Foundation
The DORA (DevOps Research and Assessment) research program identified four metrics that reliably differentiate high-performing engineering organizations from low-performing ones. These metrics have been validated across thousands of teams and multiple years of research. They're worth understanding in detail.
Deployment Frequency
How often does your team deploy to production? Elite performers deploy on demand, multiple times per day. High performers deploy between once per day and once per week. Medium performers deploy once per week to once per month. Low performers deploy less than once per month.
Why this matters: deployment frequency is a proxy for batch size. Teams that deploy frequently have smaller changesets per deployment, which means lower risk per deployment, faster feedback on whether changes are working, and shorter time between "code written" and "value delivered." Teams that deploy infrequently accumulate risk in large releases and lose the tight feedback loop.
Lead Time for Changes
How long does it take for a code commit to reach production? Elite performers: less than an hour. High performers: one day to one week. Medium: one week to one month. Low: one to six months.
Lead time is your end-to-end pipeline efficiency measure. Long lead times indicate bottlenecks: manual approval gates, slow CI/CD, large batch sizes, complex merge processes, or organizational approval chains. Reducing lead time doesn't just improve efficiency — it reduces the risk of large deployments and makes it possible to respond quickly to production issues.
Change Failure Rate
What percentage of deployments cause a production incident, require a rollback, or require a hotfix within a defined window (typically 24 hours)? Elite performers: 0–15%. High performers: 16–30%. This is an important nuance: elite performers don't have zero failures — they have a low but nonzero rate. Attempting to hit zero failures typically results in slower deployment frequency, which is the wrong trade-off.
Change failure rate tells you about your quality and testing process. High failure rates indicate insufficient automated testing, inadequate staging environments, poor deployment practices, or missing feature flagging (which allows rollback without redeployment).
Mean Time to Recovery
When an incident occurs, how long does it take to restore service? Elite performers: less than one hour. High performers: less than one day.
Mean Time to Recovery (MTTR) is a measure of your incident response capability. It's affected by observability (can you detect the problem quickly?), deployment tooling (can you roll back or forward quickly?), and organizational capability (do people know what to do?).
A critical insight from the DORA research: you should optimize for MTTR rather than for reducing failure rate to zero. Zero-failure ambitions drive risk-aversion that slows deployment frequency. High-MTTR capability gives you confidence to deploy more frequently, because failures are recoverable quickly.
The Business Outcome Layer
DORA metrics are better than velocity, but they're still internal-facing. For engineering leadership to communicate with business stakeholders, you need a layer that connects engineering activity to business outcomes.
Feature Adoption Rate
How many users actually use a feature within 30 days of it being available? This metric, tracked by product and engineering together, is a forcing function for asking the right question: are we building things people use?
Teams that track feature adoption develop a different relationship with product decisions. When you know that 40% of features shipped in the last two quarters achieved less than 5% adoption, it changes how you evaluate what goes on the roadmap. It creates accountability on both the product and engineering sides for actually solving user problems.
Time to Value
How long from user account creation (or initial purchase) to first meaningful engagement — the moment when a user has experienced the core value of the product? This is the "time to aha" metric, and it's fundamentally a collaboration between product, engineering, and design.
Tracking this metric reveals engineering problems that aren't visible elsewhere: slow initial load times that cause drop-off before the value is reached, onboarding flows that require too many steps, features that depend on data that users don't have at signup. These are real engineering problems that affect business outcomes but don't show up in velocity or DORA metrics.
Reliability vs. Customer Expectations
What does your customer-facing SLA promise, and how do you actually perform against it? This is different from internal SLOs (though those matter too). Engineering teams that track their reliability in terms of customer experience — not just uptime percentages — make different decisions about what to prioritize.
A team that knows "we had 3 incidents last quarter that affected more than 100 customers" has different conversations than a team that knows "our p99 API latency was 340ms against a 500ms SLO."
The Metrics That Are Hard to Collect But Worth It
Developer Experience (DevEx) Scores
The research is clear: developer experience and productivity are strongly correlated. Teams where developers feel like the tools work, the process makes sense, and they can do their best work ship more and ship better.
The challenge is measuring this. The most common approach is a regular (quarterly or bi-annual) developer survey using a structured framework. There are several published frameworks (SPACE, DevEx by Noda et al.) that provide validated question sets. The point isn't the specific framework — it's the practice of asking and acting on the answers.
The signal to watch: not the absolute score (every team has friction) but the trend. Is the developer experience improving or degrading over time? Are specific areas (deployment process, code review turnaround, local development environment) consistently flagged as painful?
Teams that track and act on developer experience metrics consistently outperform those that don't. This is one of the highest-ROI investments a VP Engineering or CTO can make.
Code Review Cycle Time
How long from opening a PR to final merge, and how much of that time is waiting? Review turnaround time is both a developer experience metric and a lead time driver. PRs that sit for 48 hours before getting a first review create context-switching overhead, block dependent work, and signal that code review isn't a priority.
Tracking cycle time by team (and by reviewer) reveals structural problems: teams that are under-resourced for review volume, senior engineers who are bottlenecks, review norms that require multiple rounds due to unclear standards.
A healthy cycle time goal: less than 24 hours to first review for non-draft PRs during working hours, less than 48 hours to merge for standard changes.
Incident Response Metrics Beyond MTTR
MTTR is a single number that aggregates a multi-phase process. Breaking it down gives you more actionable signal:
- Time to detect: How long from incident start to first alert? This is a function of your monitoring coverage and alert configuration.
- Time to acknowledge: How long from alert to human engagement? This is a function of your on-call process and page configuration.
- Time to diagnose: How long from acknowledgment to understanding root cause? This is a function of your observability tooling and runbook quality.
- Time to mitigate: How long from diagnosis to service restoration? This is a function of your deployment tooling and rollback capability.
Each phase has different interventions. You can't improve what you can't distinguish.
Building the Measurement Infrastructure
Start With What You Already Have
Before adding any new tooling, audit what your existing systems already capture. Your CI/CD platform tracks build times and failure rates. Your deployment system knows when things deployed and how often rollbacks happen. Your incident tracker knows when incidents started and when they were resolved. GitHub or GitLab track PR open-to-merge time and review cycles.
You can derive most of the DORA metrics from data you already have before spending money on anything new.
Choose One Dashboard, Not Five
The failure mode with metrics programs is creating an elaborate dashboard that nobody looks at. Choose the five to ten metrics that matter most for your current stage and focus, build one dashboard, make it the single source of truth for engineering performance conversations, and review it consistently.
Consistency is more important than comprehensiveness. A team that looks at five metrics every week for a year and acts on what they see will outperform a team that looks at thirty metrics in a quarterly review.
Instrument the Feedback Loop
The most important property of a metrics system is whether it changes behavior. After you've established baselines and set targets, check regularly: are decisions actually being made based on these metrics? Are engineers aware of the metrics and what they mean? Has anything changed in response to what the metrics revealed?
If the metrics aren't changing decisions, something is wrong — either you're measuring the wrong things, the metrics aren't visible enough, or there's no organizational mechanism for acting on them.
What Good Looks Like
To make this concrete: an engineering organization that has its measurement approach right looks something like this.
Leadership can answer, without looking anything up: what's our deployment frequency this quarter vs. last? What's our change failure rate trend? How long did our last three incidents take to resolve?
Product and engineering review feature adoption data monthly as a standard part of the roadmap process.
Developer experience scores are tracked quarterly, reviewed by engineering leadership, and at least one high-friction area is prioritized each quarter.
Individual engineers know their team's DORA metrics and understand how their work connects to them. They're not being evaluated on velocity.
Incidents are reviewed not just for root cause but for MTTR breakdown — which phase took longest and what would shorten it next time.
This isn't a distant aspiration. It's achievable in a quarter of focused work for most organizations. The payoff is an engineering organization that talks about what it's actually accomplishing rather than how busy it is.
Thought Parameters helps engineering leaders build the measurement and operational systems that connect engineering work to business outcomes. If you're trying to move beyond velocity metrics, let's talk.
Working on something similar?
We help engineering teams implement the practices covered in this post. First call is free.
Start a conversation →