TL;DR
My career engine keeps a file called BLOCKERS.md — the things a human has to unblock. Step (1) of its hourly prompt tells the agent to read it in full. Step (7) tells the same run to append to it. Nothing anywhere removes from it. This morning it is 486,652 bytes across 5,158 lines — call it 120,000 tokens, estimated, which is roughly 60% of a 200k context window consumed by one file before the agent has read anything else. The part a human would act on is the numbered list at the top: 14,277 bytes, 2.9% of the file, 15 items still open, and not one added since 2026-07-14. The other 97% is 110 dated sections in which the phrase "unchanged, none re-attempted" appears ten times and hCaptcha appears 207 times. 63% of the file was written in the last six days. This is not memory. It is an append-only transcript wearing memory's file name, and it is being paid for on every single run.
Yesterday I wrote about 149 throwaway scripts and a shared library nothing could import — capability the agent authored and then could not reach. This is the same engine failing in the opposite direction: a file the agent reaches for constantly, that has stopped being worth reaching for.
I found it while checking why an hourly run was slow to get started. I opened the file the run reads first.
The instruction that created the problem
Here is step (1) of the hourly prompt, verbatim from scripts/hourly-sweep.sh:
(1) Read ~/career-engine/MASTER-FRAMEWORK.md, ~/career-engine/RESUME-V2.md,
~/career-engine/pipeline.csv, and ~/career-engine/BLOCKERS.md for full
context on target roles, hard rules, and everything already queued.
And step (7), 6 lines later in the same prompt:
(7) Update pipeline.csv with every application submitted. Append a dated entry
to BLOCKERS.md ONLY for genuine 'needs live human' items — not for routine
progress.
Read it entirely. Append to it. That is the whole lifecycle. There is no step that consolidates, prunes, rotates, or closes anything, and no script in the directory does it either — the only references to BLOCKERS anywhere in code are those two prompt lines and a couple of comments in retired scripts. A launchd agent fires this every hour.
Both instructions are individually correct. "Read the blocker list before you work" is right. "Log genuine human-blocked items instead of skipping them silently" is right, and I wrote it precisely because an earlier version was skipping them silently. What is missing is the third instruction, the one that never feels urgent: something has to take things out.
What 21 days of append-only looks like
$ wc -l -c ~/career-engine/BLOCKERS.md
5158 486652
$ grep -c '^## 20' ~/career-engine/BLOCKERS.md
110
486,652 bytes. 5,158 lines. 69,907 words. 110 dated sections, 312 headings, referencing 129 distinct pass numbers running up to #158.
At the usual rough conversion of four characters per token, that is about 120,000 tokens — an estimate rather than a measurement, since I did not run a tokenizer over it. Even taken loosely, the shape of the number is the point. One file, read in full at the start of every hourly run, occupying somewhere near 60% of a 200k-token context window before the agent has looked at the resume, the framework, the 1,315-row pipeline, or a single job posting.
The growth curve is what changed my mind about how urgent this was:
The six days from 07-30 to 08-04 added 306,590 bytes — 63% of the entire file. The single busiest day, 2026-08-02, appended 75,063 bytes: more in one day than the file's first eight days combined. The engine got more productive, and its memory got proportionally more expensive to carry, with no mechanism anywhere converting that volume back down into something a run could act on.
The part a human would actually read stopped growing on day one
BLOCKERS.md opens the way you would want it to. The first line is # Blockers — need Italo, and under it is a numbered list — the concrete things that need a person: register the domain, record the proof reel, update the LinkedIn profile, approve the first outreach messages, answer the "are you an AI or a human being?" question on a Greenhouse form that the agent correctly refused to answer falsely.
That list is the entire reason the file exists. Here is its size relative to the file it sits at the top of:
| Section | Bytes | Share of file | Items |
|---|---|---|---|
| Numbered "need Italo" index | 14,277 | 2.9% | 22 (7 done, 15 open) |
| 110 dated append-only sections | 472,375 | 97.1% | — |
| Total | 486,652 | 100% | — |
Fifteen items are still open. The newest of them is dated 2026-07-14. The oldest — record the proof reel, update the LinkedIn profile — are dated 2026-07-09 and have been open for 26 days.
Nothing has been added to that index in three weeks — not because nothing needed a human, since the dated sections below are full of things that do. New blockers went into whichever dated section the run was writing at the time, and stayed there. The index froze while the file it indexes grew by a factor of thirty-three.
This is the specific failure worth naming, because it is not "the file got big." It is that the file's structure inverted. It began as a short list of human actions with supporting notes appended. It is now a long transcript of agent activity with a stale list of human actions on top. Same file, same name, same prompt line pointing at it — completely different artifact.
Most of the volume is the same blockers, restated
If the 472 KB were 472 KB of new information, this would be a storage problem and not much else. It isn't.
$ grep -c 'unchanged, none re-attempted' BLOCKERS.md
10
$ grep -c 'unchanged' BLOCKERS.md
83
$ grep -c 'hCaptcha' BLOCKERS.md
207
$ grep -c 'AHEAD' BLOCKERS.md
121
$ grep -c 'resolved' BLOCKERS.md
23
The same short roster of CAPTCHA walls — AHEAD, Saviynt, Veeva, Provectus, ThinkAhead, and eventually all of Lever — is recopied nearly verbatim at lines 936, 1031, 1077, 1139, 1227, 1288, 1340, 1418, 1568 and 1647. Each restatement is a well-behaved agent following its instructions: it read the standing blockers, confirmed they were unchanged, and honestly reported that it had not re-attempted them. Doing that ten times produces ten copies.
The ratio that matters is the last two commands. 207 mentions of the obstacle against 23 mentions of anything being resolved. A memory file whose dominant content is a restatement of what is still broken is not accumulating knowledge. It is accumulating a receipt for each time it noticed.
And the agent is paying to read all of it, every hour, to learn one sentence's worth of fact: AHEAD is still behind a CAPTCHA.
This is a context-engineering failure with a very ordinary cause. The budget wasn't blown on something exotic. It was blown on the file that was supposed to be the cheap orientation step.
Why no check caught it
The honest answer is that every check I have is scoped to a single pass, which is exactly the gap I keep rediscovering. Each hourly run did its job correctly: it read the file, it worked, it appended a truthful dated entry, it exited zero. There is no state in which that run should have failed.
A file growing 63% in six days is only visible if something compares the file to its own past. Nothing did. The same blindness produced seven scheduled agents that had failed without anyone noticing and an analytics gate reading an instrument that could not see its audience. Single-run validation cannot detect a trend, and almost every failure that actually costs me anything in this fleet turns out to be a trend.
It is worth separating this from how agent memory is supposed to work. My primer on agent memory covers the retrieval side reasonably well, but under-weights this: a memory store without a write policy — what gets summarized, superseded, deleted — degrades into a log, and a log read in full is the most expensive possible way to remember one fact.
Three rules I'm applying
1. Every file an agent is told to read in full gets a size budget, checked nightly. Not a limit that blocks the run — a threshold that reports. BLOCKERS.md gets a 40 KB ceiling, roughly ten thousand tokens, and a nightly check that prints the size and the delta from yesterday. Had that existed, it would have flagged 07-31 at 263 KB, a full week before I opened the file for an unrelated reason. The check is one wc -c and a comparison. The cost of not having it was measured in tokens on every one of the last five hundred runs.
2. Append-only is fine; append-only plus read-in-full is not. These are two separate needs and I collapsed them into one file. The transcript should stay — it is genuine history, and the dated sections are honest work. It just should not be the thing step (1) reads. The split: BLOCKERS.md becomes the short live index, capped, containing only currently-open items with a date and an owner. Everything else moves to blockers-archive/YYYY-MM.md, which no prompt loads and which the agent may grep on demand. Reading is the expensive operation, so bound what is read, not what is written.
3. A blocker entry is not closed until it leaves the file. Fifteen items have been open long enough to become scenery. Restating a blocker as "unchanged" ten times is the system telling me, ten times, that a human step is overdue — and because the restatements went into a growing transcript instead of a short list, the signal got quieter each time it repeated. Going forward an item either changes state or gets deleted with a one-line reason, and the live index carries an age column. An entry that survives fourteen days is not a blocker anymore; it is a decision I have not made, and it should read like one.
The reframe I keep coming back to on this fleet: handoff documents work because they are bounded. The template I published two days ago has a fixed section count for exactly this reason — the discipline is not in what it records, it is in what it refuses to keep. I wrote that down, then let the file next to it grow to 486 KB.
The 486 KB isn't the cost. The cost is fifteen human actions, some of them 26 days old, sitting in the 2.9% of the file that nobody — agent or human — has looked at since 2026-07-14.
FAQ
How big should an AI agent's memory file be?
Small enough that reading it in full is a cheap operation, because that is what "memory the agent loads every run" actually means. A practical ceiling for a file loaded at the start of every run is on the order of 40 KB, roughly ten thousand tokens. The file in this postmortem reached 486,652 bytes — about 120,000 estimated tokens, or near 60% of a 200k context window — while the genuinely actionable part of it was 14,277 bytes. The number itself matters less than having any ceiling at all with a nightly check that reports the size and the change since yesterday.
What is the difference between agent memory and an agent log?
Memory is curated and bounded: it holds current state, and writing to it can supersede or delete what was there before. A log is append-only and chronological: it holds what happened, and nothing is ever removed. Both are useful, and the failure here was giving one file both jobs. A log read in full at the start of every run is the most expensive available way to retrieve a small fact, because the agent pays for the entire history to learn the current state. Keep the log, keep it out of the prompt, and let the agent search it on demand.
Why do append-only agent files grow so fast?
Because every instruction points one direction. In this case step (1) of the hourly prompt told the agent to read the file and step (7) told it to append to it, and no step told it to remove anything, so the file could only grow. Growth then accelerates with the fleet's productivity: as the engine got busier, the last six days added 306,590 bytes, 63% of the total. Much of that volume is restatement rather than new information — the phrase "unchanged, none re-attempted" appears ten times and one CAPTCHA vendor is named 207 times — because a well-behaved agent that re-reads standing blockers and honestly reports them unchanged produces a fresh copy each pass.
How do I stop an agent's context file from bloating?
Split reading from writing. Keep a short, capped live file containing only currently-open items with dates and owners, and move everything historical into a dated archive that no prompt loads and the agent can grep on demand. Then add a nightly check on the live file's size and its delta from the previous day, reporting rather than blocking. In this case a 40 KB threshold would have fired on 2026-07-31, when the file passed 263 KB, roughly a week before the problem was noticed by accident.
Why didn't single-run validation catch the file growth?
Because nothing was wrong within any single run. Every hourly pass read the file, did real work, appended a truthful dated entry, and exited zero, which is correct behavior at every step. File growth is a property of the series, not of any member of it, so detecting it requires comparing the current state against the previous day's. This is the same class of gap that lets scheduled agents fail silently and analytics gates read dead instruments: single-run checks cannot see trends, and trends are where the expensive failures in agent fleets tend to live.
When should a blocker be removed from an agent's blocker list?
When it changes state, and no later than a fixed age limit. An item restated as "unchanged" across ten consecutive passes is not carrying information anymore; it is a decision that has not been made, and it should be labeled that way rather than recopied. A workable policy is that every entry either moves to resolved, gets deleted with a one-line reason, or is escalated once it passes an age threshold, with the live list carrying an age column so staleness is visible without reading anything. Fifteen items in this file are still open and the newest of them was written 21 days before the file was examined.
RUNNING AGENTS THAT KEEP THEIR OWN STATE?
The expensive failures in agent systems are rarely crashes. They are the files that quietly grow past the point of being useful while every individual run keeps exiting zero. Designing the memory boundary, the write policy, and the trend checks that catch what single-run validation cannot is the work I do. Get in touch, or see the press and hire page.
— Italo Campilii. Measured 2026-08-04 against ~/career-engine/ on my own machine: BLOCKERS.md byte- and line-counted, its 110 dated sections sized individually, its top index measured against the whole, and its repeated phrases counted with grep -c. The token figure is an estimate at four characters per token, not a tokenizer measurement. Every other command above is reproducible over any agent working directory.