01 · Operator
Who operates YouBot
- Company
- You.com
- Type
- AI search crawler
- Official docs
- you.com
02 · Behavior
What YouBot does
YouBot crawls and indexes public web pages for You.com search. It honors robots.txt, including user-agent rules and crawl-delay. It caches robots.txt for 30 minutes. Requests come from 68.67.112.0/24 and resolve to hostnames under search.you.com.
03 · Impact
Why YouBot matters for your site
If you allow it
- Pages can appear in You.com search and AI answers
- Honors robots.txt and crawl-delay
- Easy to verify by IP and reverse DNS
If you block it
- Adds crawl load without guaranteed referral traffic
- You may not want content used in AI answers
04 · Allow
How to allow YouBot
robots.txt
User-agent: YouBot
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "YouBot")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: YouBot
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "YouBot") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} YouBot [NC]
# RewriteRule .* - [F,L]05 · Block
How to block YouBot
YouBot follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: YouBot
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "YouBot")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: YouBot
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "YouBot") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} YouBot [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for YouBot
The user agent published by the operator:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; YouBot/1.0; +https://docs.you.com/youbot; env:prod) Chrome/X.X.X.X Safari/537.3607 · Verification
Is it really YouBot?
Anyone can copy a user agent. A reverse DNS lookup of a real request resolves to search.you.com, and that hostname resolves back to the same IP.
FAQ
Questions about YouBot
What is YouBot?
YouBot is You.com's web crawler. It indexes pages to provide real-time search results for You.com users.
How do I verify YouBot?
Genuine requests come from 68.67.112.0/24. Reverse DNS returns youbot-.search.you.com, matching the connecting IP.
Does YouBot respect crawl-delay?
Yes. You.com states that YouBot honors robots.txt rules, including crawl-delay settings.
Last reviewed Oct 7, 2026