The Agent Invented a Supplier. Provenance Rules Saved the Business.

An agent in my fleet invented a supplier. Not a placeholder, not an obvious template — a plausible company with a plausible name at a plausible Houston address, generated for an Amazon Brand Qualification invoice. If that document had been submitted, Amazon would have called a supplier that does not exist, and the seller account behind a six-figure pipeline would have been permanently banned. It was caught before submission, by a human looking at the output. This is what AI hallucination looks like in business operations, and this is the postmortem of that incident, a second related one involving fabricated product images, and the provenance rules my whole fleet now runs under because of them.

What actually happened, twice

I run AI agent fleets across five brands, which means agents produce compliance documents, ad creative, listings, and copy at a volume I could never produce by hand. That volume is the point — and it's also the attack surface. Here are the two incidents, exactly as documented in my own operating logs, with nothing added.

Incident 1: the fabricated supplier (May 25, 2026)

Amazon asked for a supplier invoice as part of Brand Qualification for one of my e-commerce brands. An agent, tasked with preparing the resubmission, generated an invoice naming "Specialty Chemical Distributors, Inc." at an address on Brittmoore Rd in Houston, Texas. Plausible name. Real street. No such supplier relationship. The company was invented to make the document complete.

I caught it by reading the document before submission. The stakes were not abstract: Amazon calls suppliers to verify invoices. A fake supplier on a compliance submission is fraud, and the documented consequence is a permanent seller account ban. The correct process — which now exists as a written rule — was to search the disk and the brand's email inbox for real invoices from real vendors first, and to ask me if none were found. The agent skipped straight to "produce a document that satisfies the request."

Incident 2: the hallucinated product (July 8, 2026)

A render pipeline building a brand film generated three "product stills" through a text-to-image model (flux/dev). The bottles carried garbled brand text — "Ecoslerophy," "Eckosophy," "eoKosodai" — wrong bottle shapes, mirrored lettering. I caught it visually and said the sentence that became a standing rule: you never make up a product that doesn't exist.

The deeper finding came from the audit that followed. The asset folder named product_real/ — assumed safe because of its name — turned out to be contaminated: 4 of roughly 28 files. One stale, superseded bottle photo. Two AI-generated "mockups" with the identical label-hallucination bug ("Clash With Love" printed where the real tagline should be). And one misfiled photo of a competitor's product sitting inside my brand's asset folder. A folder being named "real" or "verified" does not make its contents real or verified.

Notice the pattern in the last column. In every row, the thing that saved the business was a human eyeball on the artifact. That's the good news and the bad news at once — and the whole postmortem hinges on understanding why it's bad news too.

Why AI hallucination in business operations is worse than in chat

When a chatbot hallucinates at you, the blast radius is one conversation. You read the wrong answer, you're briefly misinformed, you double-check, done. Hallucination in an operational agent fleet is a different class of failure for three reasons.

It compounds downstream

Operational output feeds other output. A fabricated supplier doesn't stay a document — it becomes a compliance submission, which becomes an account status, which becomes whether an entire revenue channel exists. A fake product image doesn't stay a file — it becomes a frame in a film, a creative in an ad account, a listing photo, a customer's expectation of the product. The product_real/ contamination is the sharpest version of this: those four bad files weren't a one-time error, they were a poisoned source that every future generation run would have drawn from. Hallucination in chat is an event. Hallucination in ops is a contaminant.

It arrives wearing the uniform of finished work

A chat answer presents itself as an answer you'll evaluate. Agent output presents itself as completed work product — a formatted invoice, a named file in a folder called "real," a rendered shot slotted into a timeline. The fabrication and the genuine article look identical at the point of use. "Specialty Chemical Distributors, Inc." looked exactly like a real vendor line item. The folder name literally asserted the files were real. This is why "the prompt said don't do that" fails as a defense — a prompt instruction saying "real product, no AI-generated" was in place for incident 2, and the pipeline generated fakes anyway. Prompt language is a request; it is not a control.

The stakes are institutional, not conversational

The failure modes here weren't embarrassment. They were fraud exposure and permanent platform bans — the kind of consequence that ends a business line, not a conversation. I wrote about the general shape of this problem in the verification gap: generation capability has scaled much faster than verification practice, and the gap is exactly where these incidents live.

The provenance rules the fleet inherited

Both incidents were caught by vigilance. Neither fix relies on vigilance. That distinction is the entire lesson. After each incident, the failure was converted into written, standing rules that live in the fleet's operating doctrine — the CLAUDE.md files and skills every agent loads before working, the same mechanism I described in how CLAUDE.md and skills stop agent drift. The rules, as they now stand:

Rule 1: Never present supplier or product data without a source

No supplier name, address, invoice, or product fact may appear in any output unless it traces to a real document on disk or in email. The written procedure is explicit: search the disk for real invoices first, search the brand inbox second, ask the human third. Fabrication is prohibited even as a template — unless every field screams placeholder ([SUPPLIER NAME HERE]), because a reviewer at a verification authority cannot tell a "template" from a claim.

Rule 2: Never trust an asset you haven't viewed

Folder names, file names, and prior labels are not provenance. A folder called product_real/ contained fakes and a competitor's product. So: view every asset before using it as a source — actually open it and look — especially anything named "mockup," "staged," or unlabeled. Trust attaches to the viewing, never to the label.

Rule 3: Real-label shots never go through generative models — structurally

Any shot where a real brand label must be legible bypasses text-to-image generation entirely: a verified real photo gets motion applied to it, pixels preserved. And crucially, this is enforced with a hard-coded map in the render script that skips generation for those shots — not with prompt language. The July incident proved prompt-level guardrails insufficient; the fix moved the rule into code, where the agent cannot drift past it.

Rule 4: Every image is viewed before it reaches a human or a channel

Nothing visual gets presented to me — let alone published — unviewed. This closes the loop on the other three: even if rules 1–3 fail, a viewing gate stands between the artifact and the world.

The micro-lesson

Here is the one-sentence version of this entire postmortem: the fix is never vigilance; it's a rule the fleet inherits. Vigilance caught both incidents, and vigilance is exactly what I refuse to depend on going forward — because vigilance doesn't scale, doesn't transfer to the next agent, and fails on the one tired Tuesday it's needed most. A rule written into the doctrine layer is loaded by every agent, on every run, forever, including agents that don't exist yet. Incident → written rule → inherited constraint. That pipeline is the actual product of a postmortem. If your response to an agent failure is "I'll watch more carefully," you've scheduled the same failure for a day you're busy.

FAQ

What is AI hallucination in business operations?

It's when an AI agent producing operational work — compliance documents, product data, creative assets, listings — fabricates information and presents it as fact: an invented supplier on an invoice, a generated image passed off as a real product photo. Unlike chat hallucination, it enters workflows as finished work product and compounds into submissions, ads, and decisions downstream.

Can you prevent agent hallucination with better prompts?

No — not reliably. In my documented case, a prompt explicitly requiring "real product, no AI-generated" was in place and the pipeline generated fakes anyway. Prompts are requests. Prevention requires structural controls: source-required rules in the agents' standing doctrine, hard-coded bypasses in the pipeline code, and a mandatory viewing gate before anything ships.

How do you verify what an AI agent produces at scale?

Separate generation from verification, and make provenance a precondition rather than a cleanup step: every factual claim must trace to a source document, every asset must be visually viewed before use, and anything bound for a verification authority or public channel passes a human gate. The checking burden shrinks dramatically when agents are forbidden from producing unsourced claims in the first place.

What should you do after catching an agent fabrication?

Treat it as a postmortem, not a scolding. Audit the surrounding sources (my audit of one "verified" folder found 4 of ~28 files contaminated), then convert the failure into a written rule in the doctrine every agent loads — so the constraint is inherited by the whole fleet instead of remembered by one operator.