I was with a team recently that was celebrating one of its shift-left initiatives. Instead of stage-gating everything into one big, risky release, they started releasing work inside the iteration as soon as it was ready. That's a real milestone, and it's probably good for their customers and for empiricism.
A year ago, this team delivered about once a quarter. The last time I checked, they were shipping to production 30 times a day. A chart like that goes up and to the right, and it earns its own slide in the quarterly leadership review.
Over the same stretch, their incident rate climbed from roughly two a month to nine a month. That number doesn't get a slide. It gets a phone call to a developer at 2 a.m.
The metric was fine. It was standing alone.
Nothing exotic went wrong with that team. They picked one number, pushed on it, and it moved. I'm all for that. Deployment frequency is a good agile metric, DORA tracks it, and I'd want to know it for any team I'm helping.
The trouble was that the number sat there by itself, with nothing next to it to argue with it.
If you lead an engineering organization, the slide you present is the part you control, and whatever goes on that slide is what people will push. Managers love to say you can't manage what you can't measure. There's truth in that. Be careful what you measure, though, because Goodhart's Law is waiting: once a measure becomes a target, it stops being a good measure.
I'm not telling you to get rid of metrics. I want you to steer with them, and one metric alone can't do that. A couple of weeks ago, in our episode on proving AI ROI to a CFO, I paired change lead time with change failure rate so that gaming one leaves residue on the other. This week I want to take that pair off the board-meeting slide and turn it into something a team can look at every week, while the work is still happening.
Does shipping faster cost you stability?
There's a wrong lesson sitting in that opening story. It would be easy to hear it and decide that speed costs you stability. DORA's research says the opposite. Their metrics guide states that speed and stability are not trade-offs, and that for most teams the delivery metrics move together. Top performers do well across all five. Low performers do poorly across all five. The 2025 DORA report pushes the point further: its strongest team profiles are fast and stable at the same time.
So the team in my story is the exception, not the pattern. That's what makes it dangerous. A single number won't tell you when you've become the exception. You need something sitting next to it that can.
DORA's five delivery metrics (and the one most teams miss)
DORA's metrics guide lists five software delivery metrics in two groups. The throughput group covers change lead time, deployment frequency, and failed deployment recovery time. The instability group covers change failure rate and deployment rework rate.
Deployment rework rate
Rework rate is the one that made me stop. DORA added it as the fifth delivery metric in 2024, so it isn't brand new. It measures the share of deployments nobody planned: the ones you made because something broke in production. A couple of weeks ago I called that the work a team pays for twice.
If you're still saying "the four keys," you're one metric behind. I don't have all the answers either. I just try to do my research, and this one is worth adding to your vocabulary.
Two pitfalls worth a pause
The same guide lists common pitfalls, and two of them deserve your attention.
The first is setting metrics as goals. DORA's example of a bad target is a broad statement like "every application must deploy multiple times per day by year's end." Sound familiar? The guide doesn't predict what happens to a team that chases that target. It says a blanket goal like that makes gaming the metric more likely. Most of us have lived that.
The second is having one metric to rule them all. A lot of leaders go looking for the one or two numbers that tell the whole story, and in engineering that rarely works. DORA's advice is to track several metrics, including some with a healthy amount of tension between them. I like tension when it has a purpose, and that's the design I'd strive for on any product development or engineering dashboard.
DORA isn't the only one saying this
When I went looking for more background, I found DORA is just saying it the loudest right now.
The DX Core 4 is a framework from DX, a company that sells developer experience measurement. It measures four dimensions: speed, effectiveness, quality, and business impact. DX describes those dimensions as counterbalanced, because pushing hard on one of them, like speed, can hurt another. Side note: Atlassian, the company behind Jira, now owns DX.
Keep in mind who's selling what. A measurement vendor telling you to measure more things isn't a neutral party. The principle still holds up, though, and at Big Agile we care about principles. This one didn't start with DX or DORA anyway.
The cockpit, 1992
In 1992, Robert Kaplan and David Norton published "The Balanced Scorecard: Measures That Drive Performance" in Harvard Business Review. It rang a bell for me, because the balanced scorecard is what I use with managers and leaders who are new to agile and trying to make performance appraisals fit team-based work. Once you push team coordination, an appraisal built only around the individual stops making sense.
Kaplan and Norton compared a company's measures to the dials in an airplane cockpit. A pilot needs fuel, airspeed, altitude, bearing, and more, all at once. Rely on one instrument, they wrote, and it can be fatal. I've wanted to be a pilot since I was four, so that one landed. Their real point was that you read the measures together, so you can see whether one area improved at another area's expense.
They were writing for executives running companies, not delivery teams. I'm borrowing their image, not their whole scorecard. But think about my team again. They were flying on one dial, and the dial said everything was great.
Delivery metrics tell you what, not why
Here's the part that gets skipped all the time. The five delivery metrics are outcomes. They show you what's happening. They don't show you why, and I'm a "why" person.
Low deployment frequency could come from technical debt, from bureaucracy, or from burnout. The metrics alone can't tell you which, and in my experience with development teams, those are three very different conversations.
Go back to the team from the opening. Their deploy chart was right. Every number on it was true. It just couldn't tell anybody why the poor soul on call kept getting woken up in the middle of the night. To see that, you have to connect delivery data to how people actually experience the work.
The Balanced Signal Board
A balanced set needs more than one kind of signal, so I put it in three parts:
- Delivery tells you what the system did.
- Experience tells you what that cost the people inside the system.
- Outcome tells you whether a customer or the business noticed.
Together, those become what we call the Balanced Signal Board. "Scorecard" always sounded a little like grading to me, so we dropped it.
Every delivery metric on the board gets two neighbors on its row: an experience signal and an outcome signal. Every row also carries one note, which answers a single question. If someone gamed this number, which signal would move?
Here's what the two example rows I'll walk through look like on the board:
| Delivery | Experience | Outcome | Note: if gamed, what moves? |
|---|---|---|---|
| Change lead time (read beside rework rate) | Pulse question: how much of your week went to unplanned work? | Support tickets, failed checkouts, or a renewal at risk | Half-finished work shows up in rework rate first, then the pulse survey |
| Deployment frequency | After-hours calls for whoever is on call | Incidents a customer found before you did | Slicing changes into slivers: maybe the pulse survey, maybe nowhere yet |
Walking through one row
Let's use the team from the opening. I'm not saying their numbers are bad. They just make a good example.
Say their change lead time dropped from six days to one day. On its own, that's a great quarter, and somebody would celebrate that slide.
First, do what we did a couple of weeks ago and check it against another delivery metric. Put rework rate beside it. Now one in every four deploys is an unplanned fix for a production problem. (And as we talked about last week, fixing defects usually means writing more code, so don't celebrate code volume either.) Read those two together and the great quarter gets complicated. One reading: the work didn't get easier. It's going out unfinished and coming back.
That's all still the delivery column. Now let the two neighbors on the row weigh in.
The experience signal. Keep it small. Add one question to a pulse survey each sprint: how much of your week went to work you didn't plan? Say seven of nine engineers answer "most of it." Now you know something none of the delivery metrics could tell you. Most of this team's week is going to work nobody planned, and somebody is going to be upset about that. The board still doesn't tell you why. It tells you where to ask.
The outcome signal. Pick something a customer or the business would actually notice, even if it's uncomfortable. Support tickets. Failed checkouts in your cart. A renewal call that went sideways for one of your implementation or sales people. You don't need a perfect signal. You need one that someone outside the team would care about.
Now read the row: lead time, with rework rate beside it, the pulse question, and the outcome. That's one slide, and it tells a real story. The great quarter looks different now, and not necessarily worse. It lets you ask better questions while there's still time to correct course, instead of in an incident review where everyone is upset and nobody is thinking clearly.
The note: where would gaming show up?
The note on each row is a question you answer before anybody has a chance to game anything.
If someone pushes lead time down by shipping half-finished work, where does that show up first? On this team, probably rework rate, then the pulse survey. Write that down.
If someone pushes deployment frequency up by slicing changes into slivers, where does that show up? Maybe the pulse survey. Maybe nowhere yet. Write that down too. "Nowhere yet" is a legitimate answer, because it tells you where the board may be blind, and you can fix that.
You can keep the notes on the dashboard itself or alongside it. Either way, they go wherever the team looks every week.
A second row
On the second row, deployment frequency sits in the delivery column. Its experience signal is after-hours calls for whoever is on call. Its outcome signal is incidents a customer found before you did. Three signals, one row, one note.
The board won't stop anyone from gaming a number. As an empiricist, what I like is that it gives the game somewhere to show up, and somewhere you can look and ask better questions about what the data is telling you.
These pairings are our practice. You probably have your own. Treat them as a starting point, keep what works, and change what doesn't fit your team's work.
Keep the board small
If a signal on the board never changes a decision, take it off. We covered this last quarter when we looked at metrics nobody acts on. A board nobody acts on is just a nicer-looking report. Make sure your team, and the people around it, are part of reading it.
Try it with one team next week
Pick one team. Find the number they get praised for, the one they're excited about, and ask two questions about it:
- What does this number cost the people producing it?
- Who outside the team would notice if it went wrong?
Write down one signal for each answer. That's your first row. Look at all three together every week for a month. It's not a grade, so please don't use it to grade the team. Use it to learn which data moves when.
When the three disagree, take it to a retrospective and ask what changed. Or get the team together and say, "We're trying out some new signals. What would change here?"
If the first time you look at all three together feels uncomfortable, the board is working. Celebrate that. People often feel like metrics expose problems, and that's exactly their job. If your culture can't handle that, you have a different problem to solve. The sooner you find a problem, the cheaper it is to fix.
Steer with one gauge and you'll drive straight into the thing it doesn't show.
I moved through the board quickly here, so on Wednesday we'll publish the Balanced Signal Board as a starter template on the blog that you can copy and fill in for your own team. And if you want help building measurement that steers instead of scores, come work through it with us in one of our upcoming Big Agile classes and workshops.