01 · Operator
Who operates ExaUser
- Company
- Exa
- Type
- AI search crawler
- Official docs
- crawler.exa.ai
02 · Behavior
What ExaUser does
ExaUser fetches single URLs on request and does no scheduled or background crawling. It is separate from ExaSearchBot, which is Exa's crawler. It does not bypass logins, paywalls or CAPTCHAs and submits no forms. Requests are signed with Web Bot Auth (RFC 9421).
03 · Impact
Why ExaUser matters for your site
If you allow it
- Users and apps on Exa can read your current page content
- Fetches are user-initiated and limited to single URLs
- Requests can be verified cryptographically
If you block it
- You may not want your content used in AI retrieval products
- robots.txt rules may not apply to its fetches
04 · Allow
How to allow ExaUser
robots.txt
# ExaUser has no documented robots.txt token and does not read robots.txt.
# Use a Cloudflare, nginx or Apache rule instead.Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "ExaUser")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# ExaUser has no documented robots.txt token and does not read robots.txt.
# Use a Cloudflare, nginx or Apache rule instead.nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "ExaUser") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} ExaUser [NC]
# RewriteRule .* - [F,L]05 · Block
How to block ExaUser
ExaUser does not always follow robots.txt. Add a server or CDN rule if you need a hard block.
robots.txt
# ExaUser has no documented robots.txt token and does not read robots.txt.
# Use a Cloudflare, nginx or Apache rule instead.Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "ExaUser")
Action: BlockWordPress
# ExaUser has no documented robots.txt token and does not read robots.txt.
# Use a Cloudflare, nginx or Apache rule instead.nginx
# In your server { } block:
if ($http_user_agent ~* "ExaUser") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ExaUser [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for ExaUser
The user agent published by the operator:
Mozilla/5.0 (compatible; ExaUser/1.0; +https://crawler.exa.ai/exauser)07 · Verification
Is it really ExaUser?
ExaUser’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.
FAQ
Questions about ExaUser
What is ExaUser?
ExaUser is Exa's on-demand fetcher. It loads a page when a user or an Exa API app asks for its current content.
Does ExaUser respect robots.txt?
Exa states that robots.txt rules may not apply because fetches are user-initiated. To control Exa search and crawling, use ExaSearchBot rules.
How can I verify ExaUser requests?
Exa signs requests per RFC 9421. Public keys are published at crawler.exa.ai/.well-known/http-message-signatures-directory.
Last updated Oct 19, 2026