If lines of code shipped is the headline on your AI ROI slide, ask it one question before the next review: if the tools had made things worse, would this number show it? It wouldn't. Fixing defects is code too, so the count climbs either way. A number that can't show you a loss can't show you a return, and the free tool in this guide, the ROI Slide Swap, pairs each one with a number that can.
That question comes from Monday's episode, where a VP puts "lines of code shipped, up 30%" in front of a board and the CFO asks, "What would that number have looked like if the tools had actually made things worse?" The meeting and the 30% are made up for the example. The question works on any real slide your team sends upstairs.
Why an output count can't carry an ROI claim
Lines of code, pull requests opened and tickets closed all measure activity. Activity is a fine thing to watch on a team dashboard, as one signal among several. The trouble starts when one of them becomes the ROI claim, because a count built to go up has nowhere to put a bad quarter.
DORA's ROI of AI-assisted Software Development report (v. 2026.1) makes a version of the same point: adoption must be anchored in user value, and a user-centric focus directs the speed AI adds at real user problems rather than at more commits or pull requests. DORA names commits and pull requests there, not lines of code; extending it to lines of code is my reading. The report is published by Google Cloud, which sells AI products, so weigh its advice with that in mind.
The SPACE framework (Forsgren and colleagues, 2021, foundational) got there first, before AI coding tools reached today's scale. It warns that activity counts like pull requests and commits should never be used alone to judge productivity, and it uses lines of code as its example of a problematic metric. It doesn't throw lines of code out, though. It keeps it and adds counterweights, which is the move this guide makes for the ROI slide.
Take the counterweights away and put the count on the ROI slide by itself, and it turns into a target. Goodhart's Law covers what happens next, in the wording anthropologist Marilyn Strathern gave it in 1997: "When a measure becomes a target, it ceases to be a good measure." I put it this way: a number that can be gamed will be gamed the week that it goes on a slide. With lines of code, nobody even has to cheat.
The metrics that can show a loss, in plain terms
The replacements in the tool come from DORA's software delivery metrics, five measures in two groups. Throughput is how fast changes move: lead time for changes, deployment frequency, and failed deployment recovery time. Instability is how often changes go wrong: change failure rate, and rework rate, which DORA added in 2024. Recovery time sits with throughput in DORA's grouping, so the tool never uses it as a stability number.
In DORA's definition, lead time runs from the moment a change is committed to version control to the moment it's deployed in production. Change failure rate is the share of deployments that need someone to step in right away, such as a rollback or a hotfix. Rework rate, as DORA's 2025 report (State of AI-assisted Software Development) defines it, is the share of deployments that weren't planned and were made because of a problem in production. That isn't refactoring, which is normal work; it's unplanned deployments to fix something that broke.
None of these is immune to gaming, and DORA says so itself. DORA's guide to its software delivery metrics warns that setting them as goals makes it more likely teams will try to game them. I don't know of a metric that can't be gamed. What you can do is make gaming visible, by pairing numbers that pull against each other.
Lead time and change failure rate are the pair I'd start with. Push changes out faster than anyone can check them and lead time looks great, but change failure rate is where the damage shows. Play it safe and ship less, and change failure rate looks great while lead time shows the cost. Gaming one leaves residue on the other, where you can see it.
If I had to pick the harder of the two to game, it's change failure rate. Faking it means hiding a failure someone already had to step in on, and shipping less to keep it low shows up on lead time.
DORA's ROI of AI-assisted Software Development report says much the same in its own terms. It tells leaders to track change failure rate and "rework", which its value model names deployment rework rate, so faster delivery isn't paid for in stability. The same page keeps deployment frequency on its leading dashboard too, so DORA pairs counts with stability rather than throwing the counts out.
The tool: the ROI Slide Swap
Run it on the last slide your team sent upstairs. For each number, ask the CFO's question. If the answer is no, the tool flags that number as one of the easiest to game, moves it to the team dashboard, and names a flow or stability metric that could carry the ROI claim instead. It computes no ROI, and it doesn't tell you to stop tracking anything.
THE ROI SLIDE SWAP
Run it on the last slide your team sent upstairs,
with whoever builds that slide. One service at a
time.
THE TEST, for every number on the slide
If the tools had made things worse, would this
number show it?
No: flag it EASIEST TO GAME. Move it to the team
dashboard. Put a pair on the ROI slide instead.
Yes: it can stay on the ROI slide.
THE METRICS, from DORA's software delivery metrics
Lead time: committed to version control to
deployed in production.
Change failure rate: the share of deployments
that need someone to step in right away, such
as a rollback or a hotfix.
Rework rate: the share of deployments that
weren't planned and were made because of a
problem in production (DORA's 2025 report).
FIRST SWAP
Output number: lines of code shipped
Shows a loss? No. Fixing defects is code too.
Flag: EASIEST TO GAME. It rises with no one
cheating.
On the ROI slide: change failure rate and lead
time, per service, as a trend.
Stays on: the team dashboard.
SECOND SWAP
Output number: pull requests opened
Shows a loss? No. A fix opens a pull request too.
Flag: EASIEST TO GAME. Split one change into
several and the count climbs.
On the ROI slide: lead time, per service, as a
trend.
Stays on: the team dashboard, beside pull
requests waiting on a first review.
THIRD SWAP
Output number: tickets closed
Shows a loss? No. A closed bug ticket counts.
Flag: EASIEST TO GAME. Split a ticket, close two.
On the ROI slide: rework rate, per service, as a
trend.
Stays on: the team dashboard.
FOURTH SWAP
Output number: deployments shipped
Shows a loss? No. A hotfix is a deployment too.
Flag: EASIEST TO GAME as a count on its own.
On the ROI slide: change failure rate, per
service, as a trend.
Stays on: the team dashboard, where deployment
frequency is a real throughput signal.
YOUR NUMBER (copy this block for each one)
Output number: ______________________________
Shows a loss? _______________________________
Flag: _______________________________________
On the ROI slide: ___________________________
Stays on: ___________________________________
PAIRS THAT PULL AGAINST EACH OTHER
Lead time with change failure rate. Push
changes out faster than anyone can check them,
and change failure rate shows it. Ship less to
look safe, and lead time shows it.
WHAT THIS TOOL DOES NOT DO
It computes no ROI. It deletes nothing. It moves
numbers off the ROI slide, not off the
dashboard.
DONE WHEN: every number left on the ROI slide
would show it if the tools had made things
worse, and every number you moved has a home on
a team dashboard.ILLUSTRATIVE EXAMPLE. FICTIONAL. THE MADE-UP
BOARD SLIDE FROM MONDAY'S EPISODE.
BEFORE Lines of code shipped, up 30%. One big
green number, alone on the slide.
TEST If the tools had made things worse,
would it show? No. It goes up either
way.
AFTER Change failure rate and lead time for
one service, as a trend over time.
MOVED Lines of code shipped, to the team
dashboard, still watched as a signal.The four swaps are the ones I'd make first. Any number you add gets the same test, the same flag, and the same two homes: the ROI slide if it can show a loss, the team dashboard if it can't.
One note on the second swap. Last week's guide, the Queue Locator for the line in front of code review, put pull requests waiting on a first review beside pull requests opened. That pair stands as your team's operating view of where work piles up day to day. This tool is about what goes to the board as the return on the tooling, which is a different claim for a different audience.
It also isn't the Dashboard-to-Decision Audit, which sorts metrics into keep, cut or merge. Nothing here gets cut. Numbers move off the ROI slide, not off the dashboard.
How to put flow and stability in front of a board
Show lead time and change failure rate per service, as a trend over time. DORA's guide says its metrics work best applied to one application or service at a time, and that blending them across teams or whole organizations can be problematic. If your board insists on one company-wide number, give it to them and say out loud that the roll-up hides what each service is doing.
A trend also keeps one noisy quarter from becoming the story. Whether a metric moved is a different question from whether anything changed, and a board deserves the answer to the second one.
Be careful with money. I won't stand up and say faster lead time made the company money, and I'd steer you away from saying it too. The claim your slide can defend is the early part of the story: delivery got faster, steadier, or both, service by service, and here is the trend.
The cost side is the part a CFO already knows how to count. Every unplanned deployment to fix production is work the team paid for twice, and rework rate tells you how often that happened. Put it beside the trend and let finance do the arithmetic with salary numbers they already trust.
If you want a quicker gut check for any AI productivity claim, the one question that asks what a high number might hide pairs well with this tool.
Try this next week
Pull up the last slide your team sent upstairs. Find every number that only goes up, and ask the CFO's question of each one. If the answer is no, don't delete it yet; move it to a team dashboard, where it sits as a signal among other signals.
Then put change failure rate and lead time on the slide for one service, as a trend. One service is enough to start. If a number can't show you a loss, it can't show you a return. Goodhart's Law is why the count drifted onto the slide as a target, and a pair that can move both ways is how you take it back off.
This is slide work we do with leaders in our coaching engagements, one service at a time.
Bring a pair that can show a loss to your next review, and send me a note if you need help (lance@big-agile.com).