Two Gartner numbers are sitting on top of each other right now, and most people quoting one have not read the other.
The first: up to 40% of enterprise applications will ship with task-specific AI agents embedded by the end of 2026, up from under 5% in 2025. That is the number in every deck this quarter.
The second: over 40% of agentic AI projects will be canceled by the end of 2027 — from a poll of more than 3,400 organizations actively investing — because of escalating costs, unclear business value, or inadequate risk controls.
Read together, they describe one thing: a very large number of teams are about to put agents into production and then find out they have no way to tell whether the agents are doing the job. The build problem got solved in 2025. The trust problem did not.
I run the operating infrastructure of five brands by myself — content engines, e-commerce, ad pipelines, publishing, dashboards — through Claude Code and scheduled agent fleets. I hit this wall about eighteen months before the Gartner deadline, for the boring reason that I had nobody to hand a bad output to. There is no QA team downstream of me. If an agent writes something wrong at 3 a.m., it publishes wrong. So I had to build the verification layer or stop running agents. This is what I learned.
The demo-to-production gap is not a model gap
Every serious agent failure I have had was downstream of the model doing exactly what I asked.
The model didn't hallucinate the price. It used the price from the file I pointed it at, which was four months stale. The model didn't invent a product claim. It inferred one from marketing copy that was itself written by an earlier agent. The model didn't skip the step. It ran the step against an API that had silently started returning empty arrays, got nothing, and wrote a perfectly coherent summary of nothing.
This is why swapping to a better model does not fix a production agent. A better model executes a broken instruction more fluently. When Gartner lists "inadequate risk controls" as a top cancellation cause, that is the polite phrasing of what I just described: the agent worked, and nobody could tell that the work was garbage until it was downstream.
The gap between a demo and production is not capability. It is that a demo has a human watching every token, and production does not. Everything I have built since is an attempt to replace that watching human with something cheaper that never gets bored.
What I actually run: three gates, not one review
I stopped thinking about "reviewing agent output" and started thinking about gates the work has to pass before it moves. Three of them, in order of how cheap they are.
Gate one is deterministic. Before an agent's output touches anything real, code checks it. Does the file exist. Does the JSON parse. Does the word count fall in range. Does the price string match the price in the source of truth. Is there a citation for every number. These checks are unglamorous and they catch most of it. Critically, they cost nothing per run and they never disagree with themselves. If you can express a requirement as a script, never express it as a prompt.
Gate two is adversarial. For anything a script can't judge — is this argument sound, does this claim actually follow from the source — I run a second agent whose only job is to attack the first one's output. Not "review this," which produces agreeable mush. Refute this; default to rejecting if you can't verify. The generator and the critic have to be separate calls with separate instructions, because an agent asked to check its own work will find it good. That is the single highest-leverage thing I have added to my fleet.
Gate three is a human — me — but only on exceptions. Gates one and two exist so that gate three fires rarely. If I am reviewing everything, I have not built a system, I have built a job. My rule is that any output which passes one and two publishes, and anything that fails routes to me with the failure attached. On my 500-post SEO engine running on a 2-hour scheduled loop, I touch a single-digit percentage of what it produces.
The kill criteria question nobody asks first
I carried this over from systematic trading, where I test strategies walk-forward with hard kill criteria written down before a single dollar goes live. The discipline transfers cleanly, and almost nobody applies it to agents.
Before I let an automation run unattended, I answer one question in writing: what observation would make me shut this off? Not "if it seems bad." A specific, measurable trigger. If the adversarial gate rejects more than a third of output for two consecutive days, the pipeline pauses itself. If cost per accepted artifact crosses a ceiling, it pauses itself. If zero human corrections happen for a month, I go check that the gate is actually running, because a gate that never fires is usually broken rather than satisfied.
This is where I think the 40% cancellation number comes from. Teams launch agent projects with a success definition and no failure definition. So when the thing underperforms, there is no trigger to stop — just a slow quarter of nobody wanting to say it out loud, and then a cancellation that gets written up as "the technology wasn't ready." The technology was fine. The project had no brakes.
Kill criteria also do something political that matters inside a company: they convert "I have a bad feeling about the agent" into "the agent crossed the line we agreed on in March." One is an opinion and gets argued with. The other is a fact and gets acted on.
The checklist I run before any agent goes unattended
- Name the one process. One messy, specific, repeated workflow — not "marketing." If you can't name the step it replaces, you don't have a project, you have a budget line.
- Write the kill criteria first. What observation shuts this off, at what threshold, checked how often. Before you build it, while you're still honest.
- Pin the source of truth. Every fact the agent can assert traces to a file, an API, or a database you control. If it can't cite it, it can't say it.
- Script every check a script can do. Format, ranges, required fields, math, cross-references against the pinned source. Cheap, deterministic, first in line.
- Add a separate adversarial pass. A distinct call, instructed to refute rather than review, defaulting to rejection under uncertainty. Never let the generator grade itself.
- Define the exception route. What happens to a failed artifact — where it goes, who sees it, how it gets fixed. An alert nobody owns is a log line.
- Instrument cost per accepted artifact. Not cost per run. Runs that produce rejects still cost money. This is the number that tells you if the thing is worth running.
- Log the disagreements. Every time you overrule a gate, write down why. That log is your next set of rules, and it is the only real training data you own.
- Run it attended for a week. Watch it work before you let it work alone. You will find the stale file in the first two days.
- Schedule a re-read. Put a recurring date on the calendar to reopen the instructions and the kill criteria. Agent configs rot exactly like code, and nothing tells you when.
If you want more on how those instructions stay stable over months rather than drifting, I wrote about how CLAUDE.md and skills stop agent drift — that's the config layer underneath these gates. And the mechanics of running this on a timer without a human in the loop are in scheduled AI agents that work while you sleep.
Why this is an operator problem, not a platform problem
There is a reflex right now to wait for the vendor to solve this. Somebody will ship agent observability, a governance dashboard, an eval suite, and the verification problem goes away.
It won't, because the hard part isn't the plumbing. The hard part is that only you know what "correct" means for your process. No platform can tell you that a product claim needs substantiation, that this price is the one customers see, that this tone is off-brand, that this output is technically accurate and still wrong for the customer it's going to. That knowledge lives with whoever owns the outcome. The tooling can enforce the rule; it cannot author it.
Which is why the person who can do this is neither the ML engineer nor the domain expert alone. It's someone who understands the workflow well enough to define correct, and can write the harness that enforces it. Those two skills live in different departments at most companies, and the handoff between them is exactly where the 40% goes to die.
That combination is what I've spent the last two years building into every venture I run — the reason I can operate five brands' infrastructure solo isn't that the agents are good. It's that the gates are. I wrote more about that shape of work in what an AI systems builder actually does.
FAQ
Won't better models make the verification layer unnecessary?
No, and I'd argue the opposite. A more capable model expands what you can hand off, which increases the volume of unattended output — so the cost of a bad artifact getting through goes up, not down. Better models also fail more persuasively: fluent, confident, internally consistent, and wrong because the input was wrong. The failures I described earlier were all input and instruction failures, and no model upgrade touches those. The verification layer scales with autonomy, not against it.
How much does running a second adversarial agent cost?
Meaningfully less than you'd guess, because the critic reads one artifact and returns a verdict — it doesn't do the expensive generation work. In my pipelines the adversarial gate runs well under the generation cost of the artifact it's checking. The number to watch isn't gate cost in isolation, it's cost per accepted artifact. A gate that rejects a quarter of output and prevents one wrong published claim has already paid for itself many times over, and it's why I instrument that metric rather than raw spend.
Where do I start if I have no agents in production yet?
Pick the one process where you already know what correct looks like and can check it mechanically — that's usually something with a format, a range, or a source of truth to compare against. Build it with the gates from day one rather than adding them after; retrofitting verification onto a running pipeline means auditing every artifact it has already shipped. Then run it attended for a week before you set it loose. The first automation's job isn't to save time. It's to teach you where your process was ambiguous, and it always is.
— Italo Campilii. If you're building something that needs this kind of operator, get in touch.