The deploy succeeded. The HTML validated. The links resolved. The agent reported done, and by every check it could run, it was done. Then I opened the page in a real browser at real size, the way a customer would, and saw hollow wireframe boxes where product covers should be. This is the postmortem of that failure class — QA for AI-generated content that passes structural checks and fails human eyes — and the permanent gates it produced. Of everything in this build log, this is the lesson I'd hand first to any team adopting agents.
TL;DR
Agents verify what they can measure: the code ran, the file exists, the page returns 200. Customers judge what they can see: does it look finished, does it match the subject, would I buy this. Those are different tests, and my fleet failed the second one three documented times — a book-store page whose CSS-built covers collapsed into empty wireframes at real browser size, blog posts whose images depicted the wrong place entirely, and ad assets presented without anyone actually looking at them (a competitor's product selected by filename, AI-generated people with missing limbs). The durable fix was a standing doctrine, not more vigilance: render every deliverable at the size a customer sees it, view every asset with your own eyes before presenting it, and compare new work against a known-good reference. "The code ran" is not "the work is good."
The gap: structural checks vs. customer eyes
I run agent fleets across five brands, and I've written before about the verification gap — the space between "the agent says it's done" and "it's actually done." This postmortem is about a specific, sneaky corner of that gap: work that IS structurally done. Not broken. Not erroring. Deployed, valid, linked, indexed. And still a failure, because the only test that matters for customer-facing work — a human looking at it the way a customer does — never ran.
An agent's natural QA loop is made of checks it can execute: exit codes, HTTP statuses, file existence, HTML validity, maybe a DOM query for the presence of an element. Every one of those checks can pass while the page is visually broken, the image is factually wrong, or the asset is embarrassing. Nothing in curl -I tells you the hero text is whisper-sized. No linter flags a photo of Tennessee on a post about Florida. The failure modes below are all real, all documented in my operating logs, and all invisible to structural QA.
Incident 1: the page that passed every check and failed at real size
June 10, 2026. A books storefront page shipped for one of my brands. Instead of rendering real cover images, the build reconstructed each book cover as inline CSS — typography, colors, and layout approximated in markup. In the abstract, that's clever. In my head, reviewing the code, it looked fine. It deployed cleanly and the deploy was verified.
Then it was viewed at actual browser size. The mini-covers were hollow wireframes: roughly two-thirds empty space, clipped titles, text shrunk to illegibility. The catch came from one screenshot and one question from a customer's point of view: "Are they memorable? Will people want to buy this? If not, fix it!!"
Deploy-verified is not desire-verified. That sentence became doctrine. The technical lesson underneath it: fragile CSS reconstructions of visual designs drift and collapse at small sizes. The fix wasn't better CSS — it was rendering real images (headless-browser screenshot of the source design, exported to an actual JPG) so what ships is a solid, finished visual instead of an approximation that only holds together at one zoom level.
Incident 2: the image that was a false statement
A blog engine I run for a client's travel site sourced post images from whatever already existed in the site's public/images/ folder. Structurally flawless: every image tag pointed at a file that existed and loaded. Then I looked at the output as a reader would. A post about Myakka River State Park — inland Florida, alligators, a canopy walk — shipped with dollywood.jpeg. Dollywood. Tennessee. Plus a Boca Grande lighthouse and two beach-property photos. Not one image depicted the subject, and the problem was site-wide, because the sourcing logic guaranteed it.
Here's the reframe that made this a permanent rule rather than a one-off fix: a mismatched image is a factual claim that's false. It's not a style problem. It misleads the reader, wrecks the credibility the content exists to build, and hands search engines wrong signals about what the page covers. So the rule now encoded into every content engine I run is a strict sourcing ladder: (1) a genuinely correct image already on hand — with context checked, because the same lighthouse photo is right for one post and wrong for another; (2) a licence-verified public-domain or CC image of the actual subject, self-hosted with source and licence logged in a credits file; (3) no image at all. Missing is honest; misleading is not. And the alt text must describe the actual image, because alt text is also a factual claim.
Incident 3: assets presented without being viewed
The first ad build for one of my product brands compounded three unforced errors, all downstream of the same root cause — assets selected and presented without a human (or the agent) actually looking at them. First, the "product photo" pulled into the ad was a competitor's product, grabbed from a downloads folder because the filename sounded right. Second, AI-generated footage shipped with anatomy failures — a held baby with no legs or feet, and two identical "twin" children — that anyone would catch by zooming in, which nobody did. Third, the video was delivered silent, with no music bed, because no one experienced it the way a viewer would.
Every one of those files existed, loaded, and rendered. The pipeline ran end to end. And the deliverable, viewed cold, contained a rival's bottle and a child with no feet. The standing rule that came out of this is blunt: view every image with your own eyes before presenting it. Never select an asset by filename. Confirm the brand mark is actually visible and actually yours. Inspect AI-generated humans for anatomy before anything ships. And source assets from the brand's own site — where the verified, professional images live — before ever reaching into a local folder of unknowns.
Why "the code ran" is not "the work is good"
| Incident | Structural checks it passed | Customer-eye test it failed | Gate created |
|---|---|---|---|
| CSS wireframe covers | Deployed, valid HTML, elements present | Illegible and hollow at real browser size; nothing a buyer would want | Render at real size before reporting done |
| Dollywood on a Florida post | Image files existed and loaded; pages built cleanly | Images depicted the wrong place — a false factual claim | Sourcing ladder: correct image, licence-verified image, or none |
| Unviewed ad assets | Pipeline ran end to end; files rendered | Competitor's product; AI anatomy failures; silent video | View every asset before presenting; never trust a filename |
The pattern across all three: agents optimize toward the checks that exist. If your definition of done is "the build succeeded," you will get builds that succeed and pages that repel customers — because nothing in the loop ever asked the customer's question. This is the same mechanism as an agent inventing a supplier to make a document complete: the system satisfies the test it was given. The fix, in both cases, is to change what the test is.
The permanent gates: customer-eye QA as doctrine
The fix that held was not "review more carefully." Vigilance doesn't scale, doesn't transfer to the next agent, and fails on a busy Tuesday. What held was writing the customer's test into the standing doctrine every agent loads, so QA of AI-generated content runs the same gates whether I'm watching or not:
Gate 1: render at real size
Before anything customer-facing is reported done, it gets screenshotted at the size a customer actually sees it — browser viewport, phone width, marketplace thumbnail — and judged cold: Is there a solid, finished visual, a real image rather than a fragile reconstruction? Is it memorable — one concept you can describe in a sentence? Why would someone buy this? Why now? Weak answer on any of the four means iterate before delivering, not after.
Gate 2: compare against a known-good reference
New pages don't get judged in a vacuum; they get rendered next to an existing page on the same site that's already right. The reference comparison catches what absolute checks miss — off-theme styling, broken headers, disproportionate heroes, thin content — because "does this match the page that works" is a much sharper question than "is this valid." My autonomous build loops bake this in: a design-QA pass renders every new page in a real browser against a reference page before the work counts.
Gate 3: view every asset
Nothing visual reaches a customer — or even reaches me for review — unviewed. Every image opened and looked at. Brand marks confirmed. AI-generated humans inspected for anatomy. Alt text checked against what the image actually shows. Filenames and folder names trusted for nothing.
These gates cost minutes. The incidents they prevent cost credibility, and credibility is the whole product. And because they're written doctrine rather than personal habit, they ship inside the same safe-rollout process as every other change my fleet makes — inherited by agents that don't exist yet.
The micro-lesson
If your team is adopting agents, this is the transferable core: your agents' definition of done is whatever tests you gave them, and structural tests cannot see what a customer sees. So add the customer's test to the definition — render at real size, compare against a reference, view every asset — and write it where every agent inherits it. "The code ran" tells you the machine is satisfied. Only a customer-eye pass tells you a human will be.
FAQ
What is customer-eye QA for AI-generated content?
It's a QA pass that judges a deliverable the way a real customer will encounter it — rendered in a real browser at real size, viewed with human eyes — instead of relying on structural checks like build success, valid HTML, or resolving links. It exists because agents reliably pass structural checks while shipping work that is visually broken, factually mislabeled, or unpersuasive.
Why do structural checks miss these failures?
Because they test existence and validity, not meaning. A check can confirm an image file loads; it cannot confirm the image depicts the subject. It can confirm a page deployed; it cannot confirm the text is legible at phone width or that a buyer would want what's on it. Every documented incident in this postmortem passed all structural checks in place at the time.
How do you QA AI-generated content at scale without reviewing everything by hand?
Move the customer's test into the pipeline. My build loops render each new page in a headless browser, screenshot it at customer-realistic sizes, and compare it against a known-good reference page on the same site before the item counts as done. Asset rules — never select by filename, view before presenting, strict image-sourcing ladder — are written into the standing doctrine every agent loads, so the gates run on every job by default.
What should a team adopting agents do first?
Write the three gates into whatever context your agents load on every run: render at real size before reporting done, compare new pages against a reference, and view every asset before it's presented or published. Then treat each miss as a postmortem that adds a rule, not a scolding — the failure classes here each produced a permanent written gate, and none has recurred through that gate since.
Related
If you're putting agents in front of customers and want the QA doctrine — the rendering gates, asset rules, and written SOPs that keep fleet output at human standard — that's the work I do. Get in touch, or start with the verification gap.