01 · Operator
Who operates Claude-User
- Company
- Anthropic
- Type
- AI assistant fetcher
- Official docs
- support.claude.com
02 · Behavior
What Claude-User does
Claude-User fetches pages when a person asks Claude something that requires reading a website. It is triggered by user requests, not by automatic crawling. Anthropic states that disabling it prevents Claude from retrieving your content in response to user queries.
03 · Impact
Why Claude-User matters for your site
If you allow it
- Claude can read and reference your page when users ask about it
- Anthropic says blocking may reduce visibility in user-directed web search
- Requests reflect real user demand
If you block it
- You do not want Claude to fetch your pages for users
- Fewer automated requests to your server
04 · Allow
How to allow Claude-User
robots.txt
User-agent: Claude-User
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "Claude-User")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: Claude-User
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "Claude\-User") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} Claude\-User [NC]
# RewriteRule .* - [F,L]05 · Block
How to block Claude-User
Claude-User follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: Claude-User
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "Claude-User")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: Claude-User
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "Claude\-User") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Claude\-User [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for Claude-User
The operator does not publish a full user agent string. Requests contain the token “Claude-User”.
07 · Verification
Is it really Claude-User?
Anyone can copy a user agent. Real Claude-User requests come from the IP ranges published at claude.com/crawling/bots.json.
FAQ
Questions about Claude-User
Does Claude-User follow robots.txt?
Anthropic lists Claude-User among its bots that honor robots.txt, so a Disallow rule for Claude-User stops it from retrieving your pages.
What happens if I block Claude-User?
Anthropic says Claude will not retrieve your content in response to user queries, which may reduce your visibility for user-directed web search.
Is Claude-User used for AI training?
Anthropic describes Claude-User as supporting user requests; training crawling is done by ClaudeBot.
Last reviewed Oct 6, 2026