"A human will remain in the loop." That sentence is sitting in a lot of AI policies right now. Maybe yours. It sounds like a safeguard. Read it again and it starts to come apart. Which human? Which decision? At what exact moment can that person say no?
If you can't answer those three questions, you don't have a human-in-the-loop policy. You have a sentence that makes people feel better. I want to walk through why that gap is showing up everywhere this year, and how to close it before your agents outrun the people who are supposed to be watching them.
The chicken and waffles problem
Let me tell you about a restaurant this week. Nothing to do with software, so stay with me. I had two of my sons with me. One wanted chicken tenders, the other wanted waffles. The kitchen had already stopped making breakfast waffles for the day, which is fine. But the menu had a dinner item called chicken and waffles. Same chicken, same waffles, we'd ordered it before.
So I ordered it and asked them to put the chicken on one plate and the waffles on another. The server said, "I'm sorry, sir, there are no substitutions. The manager is really strict on that." I explained I wasn't asking for a substitution, just two plates instead of one. She held the line. I asked for the manager, who of course wasn't available. We left.
Think about what was actually going on with that server. She was a human in the loop. She was standing right there with full context of the situation. She simply had no authority to evaluate an exception. That is what this whole topic is about. It wasn't a failure of the person, and it wasn't a failure of the policy. The design put a human at the decision point and gave her nothing to decide with beyond fear of what the manager said.
Here's the part most oversight designs quietly ignore: exceptions always show up. When you build a process for a lot of people at scale, you have to assume the exception is coming, because it is.
Four ways human-in-the-loop breaks down
This isn't just my opinion. Jason Bloomberg at Intellyx published an analysis of why human in the loop keeps breaking down in agentic systems, and I like that he sorts the failure modes by their cause. A few are worth naming.
The rubber stamp
You have humans in the loop, but they don't have the power to fix anything. Approval turns into a formality. The signature happens; nothing changes.
Accountability laundering
Something goes wrong and everybody has plausible deniability, because a human approved it. No single person is at fault. The oversight didn't prevent the error. It just spread the blame around so the post-mortem has nowhere to land.
Governance lag
This is the one I'd worry about most. Your agent acts in seconds. Your governance runs on meetings, approvals, audits, and legal review, which take hours, sometimes days, sometimes months. An agentic system can go badly wrong while your people are still trying to get the meeting on the calendar. That is the opposite of agility.
The scaling wall
One agent making ten decisions a day, a person can watch. Forty agents making ten thousand decisions, nobody can. That model stops working long before anyone says so out loud.
Why humans can't watch a reliable system
The rubber stamp doesn't happen because people are lazy. A 2026 paper in the journal AI and Ethics on designing oversight that actually means something pulls from decades of human-factors research going back to aviation and medicine. The finding is consistent, and a little uncomfortable: humans are "poor at sustained monitoring of highly reliable systems."
Sit with that phrase. The better your system gets, the worse your humans get at watching it. If the agent is right ninety-eight times in a row, the ninety-ninth approval is a reflex. That isn't a character flaw. It's how attention works.
Aviation worked this out a long time ago. Put a pilot in front of an autopilot that almost never fails and ask them to monitor it for four hours, and their attention degrades every single time, not because they're careless, but because nothing in those four hours asks them to do anything. Anesthesiologists describe the same job as 99% boredom and 1% fear. Your AI reviewers are sitting in that same seat. Most of the AI policies I've read were written to make somebody feel better, not to actually stop anything.
Watch what happens when the lawmakers try
Nobody is going to force you to design real oversight this year. In June, the European Union pushed its high-risk AI rules, human oversight requirements included, back to December 2027. The leading state bill on human oversight here in the US is still sitting in committee. Your agents did not slow down to match any of that.
I'm not a lawyer, and none of this is legal advice, but I read legislation the way I read post-mortems. You learn a lot from watching careful people run straight into a hard problem. California's SB 833 is still pending and covers only state agencies running critical infrastructure. It would require a human to monitor the system in real time and approve any action before it executes. Then it adds an exception: if that prior review turns out to be substantially disruptive, the operator switches to periodic review instead.
Read that again. The people writing the human-oversight bill hit the scaling problem themselves and wrote the fallback right into the statute. Even in draft law, "in the loop" quietly becomes "on the loop." It's recursion at its finest, and it tells you the sentence was never going to be enough.
Treat oversight as an architecture
So stop treating oversight as one sentence in a policy and start treating it as an architecture. Different decisions call for different kinds of human judgment. Looking across all of this research, I keep coming back to four postures.
In the loop means a person approves before any action happens. On the loop means the system acts and a person is watching and can step in. Over the loop means a person sets the boundaries and reviews the outcomes in aggregate. No loop means it runs and you accept that.
Every AI decision in your organization is already in one of those four categories. Most teams have just never named it or chosen it on purpose. It's been one sentence in a policy the whole time.
Three questions that place any decision
How do you pick the right posture? Three questions do most of the work. What does this cost if we get it wrong? Can we undo it? How fast does it move? Notice that severity is only one of the three. Reversibility and speed do just as much work, and most teams never ask about either.
Take an agent that reviews pull requests and leaves comments for the team. What does a bad comment cost? A little developer time. Can you undo it? Instantly, you ignore it. How fast does it move? All day, constantly. Requiring an approval on each comment would cost more than the mistakes do. That's a job for over the loop. You sample the comments weekly and track how often developers reject them, and if the rejection rate jumps, a human goes and looks.
Now take an agent that spots anomalous behavior on an account and revokes access. What does a bad revocation cost? A customer locked out of their own money at nine o'clock on a Friday night. Can you undo it? Yes, but only if somebody is actually there to do it. How fast does it move? Seconds, and the speed is the entire point. This is where teams most often get it wrong. They see high stakes and reach for in the loop, but requiring an approval on a security response defeats the response. That one belongs on the loop with a hard requirement attached: somebody reachable in minutes, not hours, who can reverse the action without asking permission, and who is actually notified to look.
Over the loop is the posture leaders most underuse. You're not approving anything and you're not watching the stream go by. You set the boundaries, then read the pattern once a month. What did it do? Where did it drift? What changed? For a lot of reporting and summarization work, that's the honest answer.
The real test of authority: Jidoka and the Andon cord
Bloomberg points to an idea from the Toyota Production System called Jidoka. On a Toyota line, anybody can stop the line the second they see a defect. No blame, no career risk. It's built in. That is your test for real authority. Not whether a human is named somewhere, but whether that human can actually stop the line, and whether stopping it costs them anything.
That cord on the line is called the Andon cord, and psychological safety is a huge part of whether it works. If pulling the cord gets someone yelled at, you don't have oversight. You have a decoration.
A 30-minute exercise for your team
Go look at your AI workflows. They're probably fairly new. Pick one, just one, the one that touches customers or money. Get four people in a room for thirty minutes: the product owner, the engineer who built it, someone from the team that lives with the output, and whoever signs off on it today. Ask four questions and write the answers where everyone can see them.
Which loop is this in right now, honestly: in, on, over, or none? Who is the named human, a name, not a role? What signal would tell that person to step in, and where does that signal show up? And the most important one: can that person stop it without asking anyone's permission?
You're done when you have four written answers and at least one of them makes the room a little uncomfortable. Most teams end up finding the same thing. The named human exists. The signal does not. Nobody ever built the thing that would tell that person to go look.
Judgment, not oversight
Human judgment matters more right now than it ever has. This isn't really an AI story. AI just makes the gap loud. The word "oversight" is not the same word as "judgment," and most policies are still spending the first while pretending they've secured the second.
Somewhere in your company, someone is standing exactly where that server was. Full context, no authority. Go give them something to decide with.
If you want help turning this into how your teams actually operate, that's the work we do at Big Agile. Take a look at our upcoming classes and workshops, where we work through AI governance, agentic delivery, and the leadership decisions that sit underneath both.