TL;DR
Forty-two days ago I wrote that no agent of mine writes a word until a traffic pull has run. This morning I read the whole log series instead of just today's file. Fourteen consecutive daily pulls returned exactly one path — / — and never a single /blog/ URL. Over a sixty-day window: 20 pageloads total, against 33 published posts. I checked the instrument before I believed the number: the Cloudflare RUM beacon is present on 36 of 37 built pages with an identical, correct site token. The measurement isn't broken. It is structurally incapable of answering the question I gated on — no major AI crawler executes JavaScript, so a JS beacon cannot record the readership a GEO-optimized site is built for. A gate is only as honest as its instrument. Mine has been returning a confident, well-formed, useless number every morning since June, and the engine dutifully fell through to its fallback 14 times out of 14 without ever saying so.
Three days ago I wrote about an agent that failed safely for nine days and told nobody. This is the same failure mode wearing better clothes. Nothing errored. No exit code was wrong. A script ran at 06:00 every morning, hit a live API, got a valid 200, wrote a well-formed JSON file, and handed my writing agent a number that could not possibly mean what the design assumed it meant.
The difference between that postmortem and this one is uncomfortable. That one was a bug. This one was a design decision I published and recommended.
What the log series actually says
The engine writes one analytics file per day. I had been reading them one at a time — which is exactly how you miss this. Here is the series, read end to end:
$ cd ~/campilii-site && for f in engine/log/analytics-*.json; do
python3 -c "
import json,sys; d=json.load(open('$f'))
print('$f'.split('-',1)[1][:10], 'ok='+str(d.get('ok')),
'paths='+str(d.get('paths')), 'refs='+str(d.get('referrers')))"
done
2026-07-20 ok=True paths=[{'path': '/', 'count': 20}] refs=[]
2026-07-21 ok=True paths=[{'path': '/', 'count': 20}] refs=[]
2026-07-22 ok=True paths=[{'path': '/', 'count': 100}] refs=[]
2026-07-23 ok=False paths=[] refs=[]
2026-07-24 ok=True paths=[{'path': '/', 'count': 20}] refs=[]
2026-07-25 ok=True paths=[{'path': '/', 'count': 20}] refs=[]
2026-07-26 ok=True paths=[{'path': '/', 'count': 20}] refs=[]
2026-07-27 ok=True paths=[{'path': '/', 'count': 20}] refs=[]
2026-07-28 ok=True paths=[{'path': '/', 'count': 20}] refs=[]
2026-07-29 ok=True paths=[{'path': '/', 'count': 20}] refs=[]
2026-07-30 ok=True paths=[{'path': '/', 'count': 10}] refs=[]
2026-07-31 ok=True paths=[{'path': '/', 'count': 10}] refs=[]
2026-08-01 ok=True paths=[{'path': '/', 'count': 10}] refs=[]
2026-08-02 ok=True paths=[{'path': '/', 'count': 10}] refs=[]
Three things in that block should have stopped me weeks ago.
One path. Not "the blog is underperforming" — one row. The query asks for limit:25 ordered by count_DESC. Twenty-five slots available, one filled, every single day. Thirty-three posts, none of them ever appearing in a top-25 list that had twenty-four empty seats.
Referrers empty fourteen times out of fourteen. An empty array is a legitimate result. Fourteen consecutive empty arrays is a shape, and shapes are what you look for when a number might be lying.
Counts that are always multiples of ten. 20, 20, 100, 20 … 10, 10. Real traffic is not that tidy.
Step 1 of my own brief says the run should pick a topic the analytics says is growing, and fall through to a trend scan only if analytics is thin. It fell through fourteen times out of fourteen. The gate wasn't gating. It was a ceremony the engine performed before doing what it was going to do anyway.
Check the instrument before you believe the number
The tempting read is "the beacon must be broken on blog pages." I have made that assumption before and been wrong, so this morning I measured it instead.
$ tot=0; miss=0
$ for f in index.html blog/index.html blog/*/index.html press/index.html 404.html; do
[ -f "$f" ] || continue; tot=$((tot+1))
grep -q cloudflareinsights "$f" || { miss=$((miss+1)); echo "MISSING: $f"; }
done; echo "total=$tot missing_beacon=$miss"
MISSING: 404.html
total=37 missing_beacon=1
$ grep -oh '[a-f0-9]\{32\}' index.html blog/index.html blog/*/index.html | sort | uniq -c
35 1e89b1b65a464729aff5284de84ec58e
Thirty-six of thirty-seven pages instrumented. One site token across all of them, and it matches SITE_TAG in engine/pull-analytics.py character for character. The 404 page is the only gap, and a 404 page is not where 33 posts went.
So I queried the API directly, widening the window past anything the daily job looks at:
--- window 14d 2026-07-19 -> 2026-08-02
total: count 10, visits 10
/ count 10 visits 10
--- window 60d 2026-06-03 -> 2026-08-02
total: count 20, visits 20
/ count 20 visits 20
Twenty pageloads in sixty days. All of them on /. Zero on any post. The instrument is healthy, correctly installed, correctly queried, and pointed at the right property. It is reporting accurately. It is simply reporting on a population that barely overlaps with the one this site was built for.
Why a JavaScript beacon cannot see this site's readers
Cloudflare Web Analytics is Real User Monitoring. It works by shipping beacon.min.js to a browser and having that browser execute it and phone home. Everything downstream of that — the path, the referrer, the visit count — exists only if something ran the script.
AI crawlers do not run the script. Across the 2026 analyses of this question, the finding is consistent: no major AI crawler executes JavaScript. GPTBot, ClaudeBot, PerplexityBot, Bytespider and Meta's crawler fetch and read raw HTML. They will sometimes download JavaScript files without ever executing them — reported at roughly 11.5% of ChatGPT requests and 23.8% of Claude's — which makes server logs look busier than the rendered reality. Vercel's analysis of over 500 million GPTBot fetches turned up no evidence of JS execution at all. The notable exception is Google's Gemini, which reuses Googlebot's Web Rendering Service and inherits both its ability to render and its queueing delays.
Now put that beside this site's robots.txt, which I wrote deliberately:
User-agent: GPTBot Allow: /
User-agent: ClaudeBot Allow: /
User-agent: PerplexityBot Allow: /
User-agent: CCBot Allow: /
User-agent: Google-Extended Allow: /
User-agent: Applebot-Extended Allow: /
User-agent: meta-externalagent Allow: /
Every agent I explicitly invited is invisible to the instrument I gated the engine on. I optimized the site for one audience and measured it with a tool that can only see the other. The two decisions were made six weeks apart and never checked against each other.
The second failure: I can't read the instrument that would work
The obvious fix is to stop gating on client-side data and start gating on server-side request logs, which see every crawler whether it runs JavaScript or not. So I went to get them.
$ # zone campilii.com = df1513691f3caecab6383ac257550abc (active, Free Website)
$ # httpRequests1dGroups, tried with every Cloudflare token in ~/.env.acromatico
CLOUDFLARE_API_TOKEN DENIED: does not have permission
'com.cloudflare.api.account.zone.analytics.read'
CLOUDFLARE_BOT_TOKEN DENIED: does not have permission ...
CLOUDFLARE_DNS_ALL_TOKEN DENIED: does not have permission ...
CLOUDFLARE_DNS_PAGES_TOKEN DENIED: does not have permission ...
Four tokens, zero with server-side analytics read. Every one of them was scoped for the job it was minted for — deploys, DNS, RUM — and correctly so. But the cumulative effect is that the only traffic signal my unattended engine can reach is the one that cannot see its readers. This is the same shape as the capability my 6AM agent has configured but cannot open: the data exists, the account owns it, and the process that needs it at six in the morning has no path to it.
That is a credentials-design problem, and I'd rather find it this way than discover in three months that I'd been steering on the wrong number the whole time.
What I'm changing
| Gate input | Sees AI crawlers? | Sees low volume? | Reachable unattended today? | Verdict |
|---|---|---|---|---|
| Cloudflare RUM (current) | No — needs JS execution | Poorly; counts bucket to 10s | Yes | Demote to one input, never the gate |
| Zone HTTP request logs | Yes — every fetch, incl. bots | Yes, exact counts | No — all 4 tokens denied | Mint a scoped read token; then promote to primary |
| Search Console impressions | Partly — Google surfaces only | Yes, down to single impressions | Not wired here | Wire it; best early-demand signal |
| Crawler hits by user-agent | Yes — the actual GEO signal | Yes | Needs the same log access | The number that should drive this engine |
Three changes, in order:
Stop calling it a gate until it can close. A gate that has fallen through 14 times out of 14 while reporting ok: true is worse than no gate, because it produces a log line that looks like diligence. The puller now needs to distinguish "the API answered and there is no signal" from "the API answered and here is a signal" — and the run should say the first one out loud. My own 2026-06-23 post insisted the gate should announce failure loudly rather than pretend. It does that for a failed pull. It says nothing at all for a successful pull carrying an empty answer, which turns out to be the far more common case.
Mint a read-only zone analytics token. Narrowest possible scope, one zone, analytics read. That is the credential that turns crawler traffic from an article of faith into a number.
Measure the audience I actually built for. This site is written to be cited by AI engines, which means the honest early metric is crawler fetches by user-agent and, later, citations — not browser sessions. I have written the checklist for getting cited and built the engine that publishes into it. I had not, until this morning, built anything capable of telling me whether it was working.
The general lesson, stated plainly
Every automated system that makes decisions from data has two failure modes. The first is the one everyone plans for: the data pull breaks, and you get an error. The second is the one that actually gets you: the data pull succeeds, returns something well-formed, and answers a subtly different question than the one your logic assumes.
The first failure is loud and self-announcing. The second is silent, survives code review, and compounds daily — because every run that consumes it produces an artifact that looks exactly like a run that worked.
The check that catches it is not more monitoring. It is asking, of every number your automation consumes: what would have to be true for this instrument to physically record the thing I care about? For a JavaScript beacon and an AI crawler, the answer is "the crawler would have to run JavaScript," and that answer has been publicly no for the entire life of this blog. I could have known on day one. I needed 42 days and a habit of reading log series instead of log files.
If you run agents that act on data, audit your instruments the way you'd audit your code — verification is the layer nobody builds, and it applies to your inputs at least as much as your outputs.
FAQ
Why does my blog show zero traffic in Cloudflare Web Analytics?
First rule out instrumentation: confirm the beacon.min.js snippet is present on the blog templates and that its data-cf-beacon token matches the site tag you're querying. If both check out and the number is still zero, the likely answer is that your visitors aren't executing JavaScript. Cloudflare Web Analytics is Real User Monitoring — it only records sessions where a browser ran the beacon. AI crawlers, most command-line fetchers, and anything reading your raw HTML never appear. On this site the beacon was present on 36 of 37 pages with a correct token, and 60 days still showed 20 pageloads, all on the home page.
Do AI crawlers like GPTBot and ClaudeBot execute JavaScript?
Per published 2026 analyses, no major AI crawler executes JavaScript. GPTBot, ClaudeBot, PerplexityBot, Bytespider and Meta's crawler read the raw HTML response. They sometimes download JavaScript files without running them — reported at roughly 11.5% of ChatGPT requests and about 23.8% of Claude's — which can make server logs look like rendering is happening when it isn't. Vercel's review of more than 500 million GPTBot fetches found no evidence of execution. Google's Gemini is the exception because it reuses Googlebot's Web Rendering Service, inheriting both its rendering ability and its queue delays. The practical consequence is that client-rendered content, and client-side analytics, are invisible to most AI crawlers.
How should I measure traffic on a site optimized for AI search?
Use server-side signals, because they count every request regardless of JavaScript. The three worth wiring, in order of how early they show signal: crawler fetches broken out by user-agent from your CDN or origin request logs; Search Console impressions, which register long before clicks do; and citation or referral appearances from AI answer surfaces. Keep client-side RUM for what it's genuinely good at — telling you how human visitors move through the site once they arrive — but don't let it gate decisions about content that AI engines are the primary audience for.
What is analytics-gated publishing and does it still work?
It means no post gets written until a traffic pull has run and a topic has been chosen from what the data shows. The discipline is sound and I still run it. What this postmortem changes is the standard for the input: the gate must be wired to an instrument that can physically observe the audience the content targets, and it must be able to tell the difference between "no signal" and "a signal of zero." Mine returned ok: true with a well-formed empty answer for 14 consecutive days, and the run silently fell through to its fallback every time without flagging that the gate had not actually closed.
How do I get server-side analytics from Cloudflare?
Query the GraphQL Analytics API's httpRequests1dGroups dataset against your zone tag, which requires a token carrying com.cloudflare.api.account.zone.analytics.read for that zone. This is separate from Web Analytics RUM data, which uses rumPageloadEventsAdaptiveGroups and a site tag — a token scoped for one will not read the other. On this account all four existing tokens were scoped for deploys, DNS and RUM, and all four were denied zone analytics, which is why the fix is minting a new read-only token rather than broadening an existing one.
Isn't 20 pageloads in 60 days just low traffic rather than a measurement problem?
It's both, and separating them is the point. Low traffic is expected for a young site and isn't itself alarming. The measurement problem is that this instrument could not have told me otherwise: had AI crawlers been fetching every post daily, the number reported would have been identical. A metric that returns the same value whether the thing you care about is happening or not carries no information — which is what disqualifies it as a gate, independent of how much traffic the site is actually getting.
AUTOMATING DECISIONS FROM DATA?
The expensive failures in agent systems are rarely the crashes. They're the runs that succeed while consuming a number that means something other than what the logic assumes — instrument design, credential scope, and the verification layer that catches the difference. That's the work I do. Get in touch, or see the press and hire page.
— Italo Campilii. Measured 2026-08-02 from this site's own analytics log series and live API queries; the beacon audit and window comparisons are reproducible against any Cloudflare Web Analytics property.