01 · Operator
Who operates broken-link-checker
- Type
- SEO tool crawler
- Official docs
- github.com
02 · Behavior
What broken-link-checker does
It fetches HTML pages and checks every link and image they reference. It can crawl a whole site and reports links that fail. It does static HTML analysis and does not run JavaScript. By default it honors robots.txt, meta robots tags and X-Robots-Tag headers.
03 · Impact
Why broken-link-checker matters for your site
If you allow it
- Often run by site owners or agencies to audit links
- Light, polite traffic by default
If you block it
- Run by unknown third parties, not by a single company
- Robot exclusions can be switched off by the user
04 · Allow
How to allow broken-link-checker
robots.txt
User-agent: broken-link-checker
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "broken-link-checker")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: broken-link-checker
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "broken\-link\-checker") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} broken\-link\-checker [NC]
# RewriteRule .* - [F,L]05 · Block
How to block broken-link-checker
broken-link-checker follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: broken-link-checker
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "broken-link-checker")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: broken-link-checker
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "broken\-link\-checker") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} broken\-link\-checker [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for broken-link-checker
The user agent published by the operator:
broken-link-checker/0.7.8 Node.js/16.20.2 (Linux 5.4; x64)07 · Verification
Is it really broken-link-checker?
broken-link-checker’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.
FAQ
Questions about broken-link-checker
What is broken-link-checker in my logs?
It is an open-source Node.js link checker. Someone ran it against your site to find broken links.
Does broken-link-checker respect robots.txt?
Yes by default. Its honorRobotExclusions option is true unless the user turns it off.
How do I block broken-link-checker?
Add 'User-agent: broken-link-checker' with 'Disallow: /' to robots.txt. Users who disable exclusions must be blocked by user agent at the server.
Last reviewed Oct 7, 2026