01 · Operator
Who operates CriteoBot
- Company
- Criteo
- Type
- Other bot
- Official docs
- criteo.com
02 · Behavior
What CriteoBot does
CriteoBot visits pages and analyzes their content to categorize them. Criteo uses this for contextual advertising and to help publishers sell ad space. It follows robots.txt, including crawl-delay up to 30 seconds.
03 · Impact
Why CriteoBot matters for your site
If you allow it
- Supports contextual ads on your pages
- Respects robots.txt and crawl-delay
If you block it
- Brings no search visibility
- Little value if you do not sell ads
04 · Allow
How to allow CriteoBot
robots.txt
User-agent: CriteoBot
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "CriteoBot")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: CriteoBot
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "CriteoBot") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} CriteoBot [NC]
# RewriteRule .* - [F,L]05 · Block
How to block CriteoBot
CriteoBot follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: CriteoBot
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "CriteoBot")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: CriteoBot
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "CriteoBot") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} CriteoBot [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for CriteoBot
The user agent published by the operator:
CriteoBot/0.1 (+https://www.criteo.com/criteo-crawler/)07 · Verification
Is it really CriteoBot?
Anyone can copy a user agent. Real CriteoBot requests come from the IP ranges published at criteo.com/criteo-crawler/.
FAQ
Questions about CriteoBot
Does CriteoBot respect robots.txt?
Yes. Criteo states CriteoBot follows robots.txt, including crawl-delay values up to 30 seconds.
How can I verify CriteoBot?
Criteo publishes its IP ranges and ASNs on its crawler page. Check the request IP against those ranges.
Why does CriteoBot crawl my site?
It analyzes page content so Criteo can serve contextually relevant ads.
Last reviewed Oct 7, 2026