01 · Operator
Who operates MBCrawler
- Company
- Monitor Backlinks (SEOptimer)
- Type
- SEO tool crawler
- Official docs
- monitorbacklinks.com
02 · Behavior
What MBCrawler does
MBCrawler crawls pages for Monitor Backlinks, a tool for backlink monitoring and research. SEOptimer moved its main features into the SEOptimer platform. The legacy tool remains usable.
03 · Impact
Why MBCrawler matters for your site
If you allow it
- Helps site owners track backlinks to and from your site
- Part of an established SEO tool
If you block it
- No direct traffic benefit
- Feeds a commercial SEO product
04 · Allow
How to allow MBCrawler
robots.txt
User-agent: MBCrawler
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "MBCrawler")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: MBCrawler
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "MBCrawler") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} MBCrawler [NC]
# RewriteRule .* - [F,L]05 · Block
How to block MBCrawler
Start with robots.txt. If MBCrawler keeps showing up in your logs, block it at your CDN or web server.
robots.txt
User-agent: MBCrawler
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "MBCrawler")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: MBCrawler
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "MBCrawler") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} MBCrawler [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for MBCrawler
The user agent published by the operator:
MBCrawler/1.0 (https://monitorbacklinks.com/robot)07 · Verification
Is it really MBCrawler?
MBCrawler’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.
FAQ
Questions about MBCrawler
What is MBCrawler?
It is the crawler of Monitor Backlinks, a backlink monitoring tool now owned by SEOptimer.
How do I block MBCrawler?
Add a robots.txt group for "MBCrawler" with "Disallow: /", or block the user agent on your server.
Last reviewed Oct 8, 2026