01 · Operator
Who operates Meta-ExternalAgent
- Company
- Meta
- Type
- AI training crawler
- Official docs
- developers.facebook.com
02 · Behavior
What Meta-ExternalAgent does
Meta-ExternalAgent crawls the web automatically for use cases such as training foundation AI models or indexing content for Meta products. It is not triggered by users; user-requested fetches use Meta-ExternalFetcher. Meta states it respects robots.txt.
03 · Impact
Why Meta-ExternalAgent matters for your site
If you allow it
- Your content can inform Meta AI models and products
- Meta says it follows robots.txt, so you can limit it to parts of your site
If you block it
- Keeps your content out of Meta AI training
- No documented direct referral traffic
- Saves crawl bandwidth
04 · Allow
How to allow Meta-ExternalAgent
robots.txt
User-agent: meta-externalagent
Allow: /
Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "meta-externalagent")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.
WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: meta-externalagent
Allow: /
nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "meta\-externalagent") { return 403; }
Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} meta\-externalagent [NC]
# RewriteRule .* - [F,L]
05 · Block
How to block Meta-ExternalAgent
Meta-ExternalAgent follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: meta-externalagent
Disallow: /
Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "meta-externalagent")
Action: Block
WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: meta-externalagent
Disallow: /
nginx
# In your server { } block:
if ($http_user_agent ~* "meta\-externalagent") {
return 403;
}
Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} meta\-externalagent [NC]
RewriteRule .* - [F,L]
</IfModule>
06 · User agents
User agents we see for Meta-ExternalAgent
| User agent | Share | Last active | Status |
|---|---|---|---|
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Sa… |
58% | 3 days ago | Unverified |
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.… |
28% | 3 days ago | Unverified |
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Sa… |
8.4% | Yesterday | Verified |
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.… |
4.1% | Yesterday | Verified |
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.3… |
0.5% | Yesterday | Verified |
Mozilla/5.0 (compatible; Meta-ExternalAgent/1.0; +https://developers.facebook.com/docs/sharing/webma… |
0.3% | 7 days ago | Unverified |
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Sa… |
0.1% | Yesterday | Verified |
meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler) |
0% | 2 days ago | Verified |
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:147.0) Gecko/20100101 Firefox/147.0 (compatible; me… |
0% | Yesterday | Verified |
meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler) |
0% | 4 days ago | Unverified |
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Sa… |
0% | Yesterday | Verified |
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Sa… |
0% | Yesterday | Verified |
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Sa… |
0% | 2 days ago | Verified |
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.7632.6… |
0% | 2 days ago | Verified |
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.3… |
0% | 3 days ago | Unverified |
07 · Verification
Is it really Meta-ExternalAgent?
Meta-ExternalAgent’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.
IP addresses we saw from verified Meta-ExternalAgent requests
| IP range | Share | Last active |
|---|---|---|
57.141.22.47 |
13% | Yesterday |
57.141.22.10 |
13% | Yesterday |
57.141.0.8 |
5.3% | 10 days ago |
57.141.0.51 |
4.9% | 10 days ago |
57.141.0.26 |
4.9% | 10 days ago |
57.141.0.45 |
4.8% | 10 days ago |
57.141.0.52 |
4.7% | 10 days ago |
57.141.0.30 |
4.7% | 10 days ago |
57.141.0.34 |
4.6% | 10 days ago |
57.141.0.24 |
4.6% | 10 days ago |
Top 10 of 38 ranges. Always check against the operator’s current list; ranges change.
FAQ
Questions about Meta-ExternalAgent
Does blocking Meta-ExternalAgent affect link previews on Facebook?
Link previews are fetched by a different crawler, facebookexternalhit, which has its own robots.txt token.
How do I verify Meta-ExternalAgent?
Meta does not publish a JSON list; it says to get its current IP ranges by querying routes for AS32934, e.g. whois -h whois.radb.net -- '-i origin AS32934'.
How do I block Meta-ExternalAgent?
Add "User-agent: meta-externalagent" followed by "Disallow: /" to robots.txt; Meta states the crawler respects these rules.
Last updated Nov 15, 2026