Cloudflare Lets Sites Allow or Block Personal AI Agents
A new Personal Assistants view in Security Analytics shows each agent's paths and errors, and a per-agent setting can keep them off pages that run ads.
Cloudflare now shows you which personal AI agents visit your site, and you can now allow or block each supported one, or keep it off only the pages that carry your ads.
It went live on November 4, says the Cloudflare blog post on personal agents.
What you can see now
In Security Analytics, the Bot Analysis chart has a new Personal Assistants category. It sits next to the existing ones:
- Verified Bot
- Likely Human
- Likely Automated
- Automated
Under Top statistics, the Personal Assistants widget lists each agent and its request count. Click one and every other widget filters to that agent.
So you’ll see the paths it asks for, the status codes it gets and the networks it comes from. Is it stuck at your checkout?
Sites on Account Abuse Protection (still early access) get more.
Each login and sign-up shows whether an agent made it, and which one.
If an attacker uses an agent on one account, a WAF rule on its Hashed User ID blocks that account alone, and the agent keeps working for every other customer who relies on it.
Three choices per agent
Cloudflare added AI bot policies this summer. Now the Agents policy gets per-agent options.
It’s renamed AI Crawler and Agent access policies. In Security Settings, you set one overall Agents policy, then pick a different one for each supported personal agent:
- Allow (do not block)
- Block on pages with ads
- Block
Built for sites that run ads
I’d expect publishers to want that middle option. You could block agents on pages with ads overall, then allow the one agent you trust.
Your Search and Training settings don’t change.
New zones get asked during setup whether they monetize pages with ads. Then Cloudflare tailors its suggested policies to the answer, and you can change them later.
How does Cloudflare know who’s who?
Its verified bots documentation says a verified bot or agent must identify itself honestly, obey robots.txt and crawl directives, and never be caught evading what site owners ask of it.
That means one of three proofs:
- A Web Bot Auth signature
- A published IP list
- Reverse DNS
The post doesn’t say which agents count as supported, or which plans get the new analytics, and that’s the part I’d want to know before I touched a single setting on a client’s zone.
Why it matters
My read: more of your readers will send an agent to compare, summarise or buy on your site before they ever open a page on it themselves.
When Vercel split AI crawlers from assistants, it was one setting for every assistant. Cloudflare now goes agent by agent.
Do your pages earn from ads? My guess is that an agent summarising your article doesn’t show its user your ads, so blocking it on pages with ads keeps those pages for people.
Still, I wouldn’t block anything yet.
Open the Personal Assistants widget and check a week of it:
- Which agents show up, and how many requests each sends
- The paths they hit (articles, product pages, checkout)
- Any 403 or 404 codes, which mean your site is breaking for them
Run a store? An agent at your checkout is a customer.
Though one gap stays open: it can name the agent, but it may know little about the person who sent it.
Meanwhile, the identity check only works for agents that play fair. Fake Googlebots show how often bots don’t, so don’t treat every agent name as proof.
That said, we allow agents on this site. I’d start there too.
Common Questions
Can Cloudflare block AI agents only on pages that show ads?
Yes. In Security Settings, under AI Crawler and Agent access policies, Cloudflare now offers “Block on pages with ads” for the overall Agents policy and for each supported personal agent.
Where do I see which personal AI agents visit my Cloudflare site?
Cloudflare’s Security Analytics shows a Personal Assistants category in the Bot Analysis chart. Under Top statistics, the Personal Assistants widget lists each agent and its request count, and clicking one shows the paths it asks for and the status codes it gets.
Does changing the Cloudflare personal agent setting affect search or AI training crawlers?
No. Cloudflare says the Search and Training policies stay as they are. The new choices only change the overall Agents policy and the one you set for each supported personal agent.
