01 · Operator
Who operates Meta-ExternalAgent
- Company
- Meta
- Type
- AI training crawler
- Official docs
- developers.facebook.com
02 · Behavior
What Meta-ExternalAgent does
Meta-ExternalAgent crawls the web automatically for use cases such as training foundation AI models or indexing content for Meta products. It is not triggered by users; user-requested fetches use Meta-ExternalFetcher. Meta states it respects robots.txt.
03 · Impact
Why Meta-ExternalAgent matters for your site
If you allow it
- Your content can inform Meta AI models and products
- Meta says it follows robots.txt, so you can limit it to parts of your site
If you block it
- Keeps your content out of Meta AI training
- No documented direct referral traffic
- Saves crawl bandwidth
04 · Allow
How to allow Meta-ExternalAgent
robots.txt
User-agent: meta-externalagent
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "meta-externalagent")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: meta-externalagent
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "meta\-externalagent") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} meta\-externalagent [NC]
# RewriteRule .* - [F,L]05 · Block
How to block Meta-ExternalAgent
Meta-ExternalAgent follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: meta-externalagent
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "meta-externalagent")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: meta-externalagent
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "meta\-externalagent") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} meta\-externalagent [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for Meta-ExternalAgent
The user agent published by the operator:
meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)07 · Verification
Is it really Meta-ExternalAgent?
Meta-ExternalAgent’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.
FAQ
Questions about Meta-ExternalAgent
Does blocking Meta-ExternalAgent affect link previews on Facebook?
Link previews are fetched by a different crawler, facebookexternalhit, which has its own robots.txt token.
How do I verify Meta-ExternalAgent?
Meta does not publish a JSON list; it says to get its current IP ranges by querying routes for AS32934, e.g. whois -h whois.radb.net -- '-i origin AS32934'.
How do I block Meta-ExternalAgent?
Add "User-agent: meta-externalagent" followed by "Disallow: /" to robots.txt; Meta states the crawler respects these rules.
Last reviewed Oct 6, 2026