01 · Operator
Who operates Twitterbot
- Company
- X Corp. (formerly Twitter)
- Type
- Social preview bot
- Official docs
- web-wp.archive.org
02 · Behavior
What Twitterbot does
When a link is posted, Twitterbot requests the page and reads its card markup. It also fetches the referenced images. Twitter's documentation states that card content is cached for 7 days after a link is first posted.
03 · Impact
Why Twitterbot matters for your site
If you allow it
- Shared links show a title, description and image on X
- Blocking it removes cards and thumbnails from your links
If you block it
- You do not want your pages previewed on X
- You want to limit requests from social platforms
04 · Allow
How to allow Twitterbot
robots.txt
User-agent: Twitterbot
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "Twitterbot")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: Twitterbot
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "Twitterbot") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} Twitterbot [NC]
# RewriteRule .* - [F,L]05 · Block
How to block Twitterbot
Twitterbot follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: Twitterbot
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "Twitterbot")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: Twitterbot
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "Twitterbot") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Twitterbot [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for Twitterbot
| User agent | Share | Last active | Status |
|---|---|---|---|
Twitterbot/1.0 |
60% | 7 days ago | Unverified |
Twitterbot/1.0 |
40% | 2 days ago | Verified |
07 · Verification
Is it really Twitterbot?
Twitterbot’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.
IP addresses we saw from verified Twitterbot requests
| IP range | Share | Last active |
|---|---|---|
199.16.157.182 |
50% | 4 days ago |
199.16.157.180 |
25% | 23 days ago |
199.16.157.181 |
25% | 2 days ago |
Always check against the operator’s current list; ranges change.
FAQ
Questions about Twitterbot
Does Twitterbot respect robots.txt?
Yes. Twitter's card documentation states that pages or images blocked by robots.txt will not show a card or thumbnail.
How do I allow Twitterbot but block other crawlers?
Add a group with User-agent: Twitterbot and an empty Disallow line. Then add User-agent: * with Disallow: / for all other bots.
How long does X cache link previews?
Twitter's documentation states that card content is cached for 7 days after the link is first posted.
Last updated Oct 7, 2026