discobot User Agent: What It Is & Should You Block It

01 · Operator

Who operates discobot

Company: Discovery Engine
Type: Search engine crawler
Official docs: discoveryengine.com

02 · Behavior

What discobot does

discobot crawled web pages for the Discovery Engine search index. User agent databases list versions 1.0, 1.1 and 2.0. We found no current activity or documentation.

03 · Impact

Why discobot matters for your site

If you allow it

Historically indexed pages for search

If you block it

Operator site no longer reachable
No current search traffic benefit

04 · Allow

How to allow discobot

robots.txt

User-agent: discobot
Allow: /

Cloudflare

# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "discobot")
Action:     Skip › All Super Bot Fight Mode rules

# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.

WordPress

# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: discobot
Allow: /

nginx

# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "discobot") { return 403; }

Apache

# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} discobot [NC]
# RewriteRule .* - [F,L]

05 · Block

How to block discobot

Start with robots.txt. If discobot keeps showing up in your logs, block it at your CDN or web server.

robots.txt

User-agent: discobot
Disallow: /

Cloudflare

# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "discobot")
Action:     Block

WordPress

# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: discobot
Disallow: /

nginx

# In your server { } block:
if ($http_user_agent ~* "discobot") {
    return 403;
}

Apache

# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} discobot [NC]
RewriteRule .* - [F,L]
</IfModule>

06 · User agents

User agents we see for discobot

The user agent published by the operator:

Mozilla/5.0 (compatible; discobot/2.0; +http://discoveryengine.com/discobot.html)

07 · Verification

Is it really discobot?

discobot’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.

FAQ

Questions about discobot

Is discobot still active?

We could not reach its documentation page at discoveryengine.com. We found no sign of current activity.

Should I block discobot?

Blocking it is low risk. Recent requests with this name may come from other clients reusing the string.

discobot was Discovery Engine’s search engine crawler