01 · Operator
Who operates Google-Agent
- Company
- Type
- AI assistant fetcher
- Official docs
- developers.google.com
02 · Behavior
What Google-Agent does
Google-Agent visits pages when a user asks a Google-hosted agent to do a task. It has desktop and mobile Chrome-based user agents. Google is experimenting with Web Bot Auth for it, using the identity agent.bot.goog.
03 · Impact
Why Google-Agent matters for your site
If you allow it
- Requests come from real users asking an agent to act on your site
- Blocking may stop users from completing tasks through Google's agents
If you block it
- As a user-triggered fetcher it generally does not follow robots.txt
- Agents may interact with forms and pages automatically
04 · Allow
How to allow Google-Agent
robots.txt
# Google-Agent has no documented robots.txt token and does not read robots.txt.
# Use a Cloudflare, nginx or Apache rule instead.Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "Google-Agent")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# Google-Agent has no documented robots.txt token and does not read robots.txt.
# Use a Cloudflare, nginx or Apache rule instead.nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "Google\-Agent") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} Google\-Agent [NC]
# RewriteRule .* - [F,L]05 · Block
How to block Google-Agent
Google-Agent has been reported to ignore robots.txt. Block it at your CDN or web server as well.
robots.txt
# Google-Agent has no documented robots.txt token and does not read robots.txt.
# Use a Cloudflare, nginx or Apache rule instead.Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "Google-Agent")
Action: BlockWordPress
# Google-Agent has no documented robots.txt token and does not read robots.txt.
# Use a Cloudflare, nginx or Apache rule instead.nginx
# In your server { } block:
if ($http_user_agent ~* "Google\-Agent") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Google\-Agent [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for Google-Agent
The user agent published by the operator:
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Google-Agent; +https://developers.google.com/crawling/docs/crawlers-fetchers/google-agent) Chrome/W.X.Y.Z Safari/537.3607 · Verification
Is it really Google-Agent?
Anyone can copy a user agent. Real Google-Agent requests come from the IP ranges published at developers.google.com/static/crawling/ipranges/user-triggered-agents.json. A reverse DNS lookup of a real request resolves to gae.googleusercontent.com</code>, <code>google.com, and that hostname resolves back to the same IP.
FAQ
Questions about Google-Agent
What is Google-Agent?
Google-Agent is a Google user-triggered fetcher. Agents hosted on Google infrastructure use it to browse and act on websites for users.
Does Google-Agent obey robots.txt?
Google says user-triggered fetchers generally ignore robots.txt because a user requested the fetch. It lists no robots.txt token for Google-Agent.
How do I verify Google-Agent?
Check the source IP against Google's user-triggered-agents.json list, or run a reverse DNS lookup as described in Google's verification guide.
Last updated Oct 14, 2026