The Safe-Rollout Playbook: Letting Agents Ship Without Letting Them Break Production

Every operator running AI agents against real infrastructure eventually writes a safe-rollout doctrine. The only question is whether you write it before the incident or after. I wrote most of mine after — which means every rule in this playbook maps to a specific failure I either suffered or watched almost happen across the five brands' worth of production systems I run solo with agent fleets.

This post is that doctrine, translated for hiring managers and operators asking the question enterprises actually care about: how do you let agents ship continuously without letting them break production? Not in theory — in standing rules that scheduled, unsupervised agents obey every day.

The framing that makes the rules stick: name the failure

Governance documents fail when they're lists of virtues ("be careful," "test thoroughly"). They work when every rule is the tombstone of a specific failure mode. So that's how I keep mine, and that's how I'll present each rule here: the rule, then the failure it exists to prevent.

The rest of the post walks each row.

Rule 1: Stage changes, one property at a time

When one fleet serves multiple properties — in my case, daily content and infrastructure jobs across five brands — the tempting move is to roll a change everywhere at once. It's the same script, after all.

That's precisely the failure surface. A defect deployed to one property is an incident; the same defect fanned out to five properties simultaneously is a crisis, and you find out about all five at the same moment. So the doctrine is boring on purpose: changes land staged, on one property, and the next property only gets the change after the first one is verified live. Pace is part of the same rule — my daily engines are capped at one new post per site per day and small queued batches for backfill work, enforced in code with a persisted queue, not by good intentions. A site that publishes 46 posts in an hour looks exactly like what it is, and everything that looks automated eventually gets treated like spam. Nothing appears abruptly, anywhere.

The deeper principle: rollout speed should be set by your verification capacity, not by your automation capacity. Agents can deploy infinitely fast. You can only check so fast. The slower of the two is your real shipping speed.

Rule 2: "Done" means a live 200, not a green log

This is the rule I'd put first if I could only keep one. No change is declared done until the live URL is fetched — an actual curl against the production address returning an actual HTTP 200, with the expected content present. Not "the build passed." Not "the agent said it deployed." The thing, live, observed.

The failure this prevents is the one I've written about at length in the verification gap in production agents: agents are fluent, confident, and structurally incapable of being embarrassed. On one multi-agent run against a shared working tree, every one of the agents self-reported success, and every agent's own build passed — while roughly a hundred file edits never actually landed on disk. The summaries were glowing. The filesystem disagreed. Only direct verification — grep, git diff, live fetch — caught it.

So the standing rule for every fleet I run: an agent's report of its own work is a claim, never evidence. Evidence is the live system doing the thing. This applies with extra force to anything a link points to — a URL goes into an email only after it is both published and returning 200, because "the page I linked doesn't exist" is a failure a recipient experiences before you do.

Rule 3: Agents never touch prod branches or deploy scripts

On protected properties, agents in my fleets are barred from the entire class of branch-mutating git commands: no checkout, no switch, no merge, no reset, no stash, no clean. They get git status and git diff for self-inspection. Commits belong to the orchestrator. Deploys belong to a human-run script, or on client infrastructure, to the client's own promote step.

The failure that carved this rule into stone: a fifteen-agent workflow editing files in a shared, non-isolated working tree. Mid-run, the reflog later showed, something executed a git checkout main, a fast-forward merge, and a checkout back — almost certainly one agent running a git command nobody had explicitly forbidden, while its sibling agents still had uncommitted edits sitting in that same tree. The branch switch silently discarded their in-progress work. No commit history was lost — I checked the reflog and dangling commits to confirm — but real work vanished and had to be redone.

Two lessons compounded from that day. First, "the agent has no reason to do X" is not a control; only "the agent is explicitly forbidden from X, in its standing context" is. Second, if agents genuinely need isolation from each other's concurrent edits, give them real isolation — separate worktrees — instead of trusting a polite division of files. This is the same philosophy as my manual-first permission model: capabilities start at zero and are granted one observed, justified action at a time. Branch mutation and deployment simply never make the granted list on anything protected.

Rule 4: Drafts-only for anything outbound

Any agent-produced artifact that would leave the building — an email, a social post, anything addressed to a human who isn't me — stops at draft. Always. The agent composes; a human sends.

The failure this prevents needs no incident story, because the whole point is that it's the one failure category you cannot roll back. A broken page can be redeployed in minutes. A wrong email sitting in a customer's inbox is permanent. Every other rule in this playbook exists because production mistakes are recoverable with speed and honesty; outbound mistakes are the exception, so they get the strictest gate: a human between the draft and the send button, with no automation pathway around it. My email tooling literally has no agent-accessible send path for external recipients — the gate is structural, not behavioral.

This is also where I'd push back on the fear I hear from enterprise teams — that agent adoption means agents talking to customers unsupervised. It doesn't have to. You can capture nearly all of the leverage (research, drafting, personalization, queue management) while keeping the send action human. The draft/send boundary is the cheapest, highest-value control in the whole playbook.

Rule 5: Preview-then-promote on client infrastructure

Everything above tightens further the moment the infrastructure belongs to a client. On my own properties, a mistake costs me a scramble. On a client's property, it costs trust — and trust doesn't redeploy.

So on client infra the doctrine is: agents build against a preview deployment, the preview is verified live (same curl-200 standard), and promotion to production is a separate, deliberate step — ideally one the client or I execute by hand. Agents never hold the promote lever. On the most sensitive properties, the rule goes all the way to local edits only: agents modify files, a human runs the deploy script, full stop.

The related discipline is honest logging. My scheduled jobs report what they skipped rather than faking success, and when a job lacks the data access to make an evidence-based decision, the log says so instead of implying it decided on data it never had. A rollout process you can't audit honestly isn't a safe rollout process — it's a quiet one, which is worse.

Why scheduled agents raise the stakes

All five rules exist because my agents don't run while I watch — they run on schedulers at six in the morning. Unattended operation changes the math: there is no human in the loop at execution time, so every control has to be in the standing rules, the permissions, and the verification gates before the run starts. I learned the supporting lesson the hard way when a scheduler PATH issue killed my launchd jobs silently for days — "armed" is also a claim, and it also requires a verified end-to-end run before you believe it.

That's the shape of the whole playbook, really. Agents make claims: "I deployed it," "I'm scheduled," "I finished the edits," "this is ready to send." Safe rollout is the discipline of converting every claim into evidence before it's allowed to matter — and of structurally removing the claims (branch mutation, outbound sends, prod promotion) whose failure you can't afford to discover after the fact.

FAQ

What does "deploying AI agents safely to production" actually require?

Five standing controls: staged changes rolled to one property at a time, live HTTP verification before any change is declared done, a hard prohibition on agents mutating git branches or running deploy scripts on protected properties, drafts-only handling for all outbound communication, and preview-then-promote on any infrastructure you don't own. Each control should be written down in the agents' standing context, not held in your head.

Should agents ever be allowed to deploy directly to production?

On low-stakes properties you own, with a verified rollback path — sometimes. On protected or client properties — no. The promote step stays human. The useful reframe: agents can do 95% of the shipping work while a human holds the last 5%, and that last 5% is where nearly all the irreversible risk lives.

How do you stop an agent from claiming work is done when it isn't?

Never accept the agent's summary as evidence. Verify the actual artifact: fetch the live URL and check for 200 plus expected content, grep the filesystem for the edits, diff the tree. Self-reported success with zero real changes on disk is a failure mode I've caught more than once — direct verification is the only reliable counter.

Why forbid git branch commands instead of just telling agents to be careful?

Because "no reason to do it" is not a control. A multi-agent run on a shared working tree lost around a hundred in-progress edits when something switched branches mid-run — an action no agent was explicitly forbidden from taking. Explicit prohibitions in standing context, or true worktree isolation, are the only versions of "careful" that survive unsupervised operation.

Does this level of control slow shipping down?

It slows rollout, deliberately, to the speed of your verification capacity — and that's the point. The work itself (building, drafting, editing) runs at full agent speed. What the playbook removes is the false speed of unverified deploys, which is always repaid later with interest during the incident.

— Italo Campilii. If you're building something that needs this kind of operator, get in touch.