01 · Operator
Who operates spbot
- Type
- SEO tool crawler
- Official docs
- openlinkprofiler.org
02 · Behavior
What spbot does
spbot crawls web pages and records the links between sites. The data feeds the backlink analysis of OpenLinkProfiler.org. The crawler supports the Crawl-delay directive.
03 · Impact
Why spbot matters for your site
If you allow it
- Your links appear in a public backlink database
- Obeys robots.txt and Crawl-delay
If you block it
- No direct traffic benefit
- Uses server resources for third-party SEO data
04 · Allow
How to allow spbot
robots.txt
User-agent: spbot
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "spbot")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: spbot
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "spbot") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} spbot [NC]
# RewriteRule .* - [F,L]05 · Block
How to block spbot
spbot follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: spbot
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "spbot")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: spbot
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "spbot") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} spbot [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for spbot
The user agent published by the operator:
Mozilla/5.0 (compatible; spbot/5.0.3; +http://OpenLinkProfiler.org/bot )07 · Verification
Is it really spbot?
spbot’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.
FAQ
Questions about spbot
What is spbot?
spbot is the crawler of OpenLinkProfiler.org. It gathers inbound link data for backlink analysis.
How do I block spbot?
Add "User-agent: spbot" and "Disallow: /" to robots.txt. The operator states that the crawler obeys robots.txt.
Can I slow spbot down?
Yes. spbot supports the Crawl-delay directive, for example "Crawl-delay: 10".
Last reviewed Oct 8, 2026