01 · Operator
Who operates BrightEdge Crawler
- Company
- BrightEdge
- Type
- SEO tool crawler
- Official docs
- support.brightedge.com
02 · Behavior
What BrightEdge Crawler does
The crawler gathers data for Site Audit and Content IQ, which check on-page SEO and technical issues. It also feeds on-page recommendations and SearchIQ. BrightEdge customers schedule crawls, for example weekly or monthly. BrightEdge uses more than one user agent.
03 · Impact
Why BrightEdge Crawler matters for your site
If you allow it
- Required if you or your agency use BrightEdge
- Feeds SEO audits of your own site
If you block it
- No benefit if you are not a BrightEdge customer
- Competitors' audits may add crawl load
04 · Allow
How to allow BrightEdge Crawler
robots.txt
User-agent: BrightEdge
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "BrightEdge")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: BrightEdge
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "BrightEdge") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} BrightEdge [NC]
# RewriteRule .* - [F,L]05 · Block
How to block BrightEdge Crawler
BrightEdge Crawler follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: BrightEdge
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "BrightEdge")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: BrightEdge
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "BrightEdge") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} BrightEdge [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for BrightEdge Crawler
The user agent published by the operator:
BrightEdge Crawler/1.0 (crawler@brightedge.com)07 · Verification
Is it really BrightEdge Crawler?
BrightEdge Crawler’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.
FAQ
Questions about BrightEdge Crawler
What is BrightEdge Crawler?
It is the crawler of the BrightEdge SEO platform. It collects data for site audits and on-page recommendations.
How do I block BrightEdge Crawler?
BrightEdge documents the rule 'User-agent: BrightEdge' with 'Disallow: /' in robots.txt.
Is BrightEdge Crawler verified?
Cloudflare Radar lists BrightEdge Bot as a verified bot. BrightEdge does not publish its IP ranges.
Last reviewed Oct 7, 2026