Mail.RU_Bot User Agent: What It Is & How to Block It

01 · Operator

Who operates Mail.RU_Bot

Company: Mail.Ru Group
Type: Search engine crawler
Official docs: go.mail.ru

02 · Behavior

What Mail.RU_Bot does

Mail.RU_Bot crawls web pages for the search service of Mail.Ru Group. A second variant identifies as Mail.RU_Bot/Fast/2.0. Both link to go.mail.ru/help/robots.

03 · Impact

Why Mail.RU_Bot matters for your site

If you allow it

Indexes your site for Mail.ru search
Can bring visitors from Russian-speaking users

If you block it

Little value for sites without a Russian audience
Adds crawl load

04 · Allow

How to allow Mail.RU_Bot

robots.txt

User-agent: Mail.RU_Bot
Allow: /

Cloudflare

# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "Mail.RU_Bot")
Action:     Skip › All Super Bot Fight Mode rules

# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.

WordPress

# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: Mail.RU_Bot
Allow: /

nginx

# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "Mail\.RU_Bot") { return 403; }

Apache

# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} Mail\.RU_Bot [NC]
# RewriteRule .* - [F,L]

05 · Block

How to block Mail.RU_Bot

Start with robots.txt. If Mail.RU_Bot keeps showing up in your logs, block it at your CDN or web server.

robots.txt

User-agent: Mail.RU_Bot
Disallow: /

Cloudflare

# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "Mail.RU_Bot")
Action:     Block

WordPress

# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: Mail.RU_Bot
Disallow: /

nginx

# In your server { } block:
if ($http_user_agent ~* "Mail\.RU_Bot") {
    return 403;
}

Apache

# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Mail\.RU_Bot [NC]
RewriteRule .* - [F,L]
</IfModule>

06 · User agents

User agents we see for Mail.RU_Bot

The user agent published by the operator:

Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)

07 · Verification

Is it really Mail.RU_Bot?

Mail.RU_Bot’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.

FAQ

Questions about Mail.RU_Bot

What is Mail.RU_Bot?

It is the search engine crawler of Mail.Ru Group. It indexes pages for the Mail.ru search service.

How do I block Mail.RU_Bot?

Add a robots.txt group for "Mail.RU_Bot" with "Disallow: /", or block the user agent on your server.

Mail.RU_Bot is Mail.Ru Group’s search engine crawler