Why Most AI Agent Pilots Die at the Handoff

Most AI agent pilots don't die because the model gets something wrong. They die at the moment the output lands in someone's queue and nobody is quite sure what to do with it, who owns it, or whether it can be trusted. The demo worked. The pilot dashboard looked healthy. Then, slowly and without anyone announcing it, the work drifts back to being done by hand.

I've spent more than 16 years in operations, and for the last few I've been designing GenAI and agentic AI systems inside a working service organisation. The pattern is consistent enough that I've stopped treating it as bad luck. AI pilots stall at handoffs, at exceptions and at ownership. Very rarely at the model.

The model is the easy part

This sounds backwards, so let me explain.

When a team starts an agent pilot, almost all the attention goes to the model. Which one to use. How to prompt it. Whether it hallucinates. Those questions matter, but they are the questions you can answer in a sandbox. The hard questions only show up when the agent meets a real process with real people on either side of it.

What happens when the agent is unsure? Who picks up the case it can't handle? Where does the record of what it did actually live? How do you know, three weeks later, why it made a particular decision?

Lean Six Sigma trained me to look at the whole value stream rather than the individual step. Waste rarely sits inside a task. It sits between tasks: in waiting, in rework, in unclear handoffs, in work that bounces back because nobody defined what "done" looks like. An agent that cuts a ten-minute task to twenty seconds looks impressive in isolation. If its output then waits half a day for a review nobody owns, the customer sees no difference at all. You've automated a step and left the process untouched.

That is why so many pilots report strong local numbers and then fail to scale. They optimised the wrong thing.

Six principles I use instead

None of these are new ideas. Most come straight from operations discipline. What's new is applying them to systems that include a probabilistic component.

1. Map the handoffs before you build anything

Before choosing a tool, draw the current process end to end and mark every point where work changes hands: person to person, system to system, team to team. Those are your risk points. If you can't say who receives the agent's output and what they do next, you aren't ready to build. A simple SIPOC or swimlane map does more for an agent pilot than another week of prompt tuning.

2. Let the LLM interpret; let deterministic tooling own state

Language models are very good at reading messy inputs: classifying a request, extracting fields, summarising context, drafting a response. They are poor at being the system of record. So I keep the two apart. The model interprets. Workflow tools, databases and APIs hold the state: what stage a case is at, who owns it, what has been approved. Every step gets logged so it can be replayed. When something goes wrong, and it will, you can see exactly what the model was given, what it returned and what the workflow did with it. Without that, governance is guesswork.

3. Route with small decisions and confidence thresholds

Not every case should go through the same path. I prefer small, narrow decision steps over one large agent trying to do everything. Each step answers one question and returns a confidence level. Above the threshold, the work flows on automatically. Below it, the case goes to a human. Thresholds are a dial you can turn as evidence builds, which is far easier to defend to a sceptical stakeholder than "the agent decides".

4. Put evaluation gates inside the flow

Quality checks that happen after the fact tell you what went wrong last month. Evaluation gates placed inside the workflow stop bad output before it reaches a customer or the next team. A gate can be a rule, a second model check, a sample sent for human QA, or a combination. The point is that quality is designed in, the same way it would be on any well-run production line, rather than inspected at the end.

5. Name a human owner for every exception

"The team will handle exceptions" means nobody will. Every exception path needs a named owner, a clear trigger and an expected response time. This is unglamorous work and it's where most pilots quietly break. Exceptions aren't a sign the agent failed. They are a normal, predictable part of the process, and they need the same design attention as the happy path.

6. Measure end-to-end cycle time and quality against a baseline

Task speed is the most tempting metric and the least useful. Before the pilot starts, capture a baseline for the full process: total cycle time, first-pass quality, rework rate. Then measure the same things afterwards. If end-to-end cycle time hasn't moved and quality hasn't improved, the pilot hasn't worked, however fast the agent is. This one discipline would save a lot of organisations a lot of money.

Where this comes from

I'm wary of articles that lean too heavily on the author's own results, so I'll keep this brief. At Prezent, an AI company where I'm now Director, Operations Excellence, I built and scaled service operations from zero to more than 500 members. Over that period, service quality moved from 76% to above 99% through Lean Six Sigma process redesign, QA and standardised SOPs. Separately, AI-led workflow automation using tools such as n8n, Zapier, the Claude API and Asana AI Studio delivered productivity gains of more than 300%.

I share those figures for one reason. The quality gains came from process discipline, and the productivity gains came from automation built on top of that discipline. In my experience the order matters. Automating a process you haven't stabilised mostly gets you faster chaos.

A quick checklist before your next pilot

If you're about to launch an agent pilot, or rescue one that's stalling, run through these:

• Have we mapped the end-to-end process and marked every handoff?
• Is it clear what the model interprets and what the workflow tooling owns?
• Is every agent action logged and replayable?
• Do routing decisions use confidence thresholds, with a defined path below them?
• Are evaluation gates built into the flow, not bolted on afterwards?
• Does every exception path have a named human owner and a response time?
• Do we have a baseline for end-to-end cycle time and quality?
• Will we judge success on those end-to-end measures rather than task speed?

If you can't tick most of these, the model is not your biggest risk.

Over to you

I'm curious where you've seen agent pilots get stuck. Was it the model, or was it something between the steps: a handoff, an exception nobody owned, a metric that measured the wrong thing? I'd like to hear what you found, and what you changed.

Mohit Bhatnagar is Director, Operations Excellence at Prezent and an AI Operations & Transformation Consultant based in Gurugram. A Lean Six Sigma Black Belt, he helps teams turn AI pilots into dependable operations. More at mohitbhatnagar.com.