01 · Operator
Who operates CompanyMapBot
- Company
- Optional LTD (CompanyMap)
- Type
- Scraper
- Official docs
- companymap.co.uk
02 · Behavior
What CompanyMapBot does
CompanyMapBot works out which website belongs to each company in the Companies House register. The data supports a map of UK companies. It reads the visible text of pages, usually the homepage and at most about 9 pages per site. Requests to a site are sequential, and a domain is re-checked a few times a year at most.
03 · Impact
Why CompanyMapBot matters for your site
If you allow it
- It fetches very few pages per site
- It honours robots.txt, including User-agent: * rules
- Your company may be correctly linked to its website
If you block it
- It brings no search traffic
- No fixed IP list is published for verification
04 · Allow
How to allow CompanyMapBot
robots.txt
User-agent: CompanyMapBot
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "CompanyMapBot")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: CompanyMapBot
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "CompanyMapBot") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} CompanyMapBot [NC]
# RewriteRule .* - [F,L]05 · Block
How to block CompanyMapBot
CompanyMapBot follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: CompanyMapBot
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "CompanyMapBot")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: CompanyMapBot
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "CompanyMapBot") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} CompanyMapBot [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for CompanyMapBot
The user agent published by the operator:
Mozilla/5.0 (compatible; CompanyMapBot/0.1; +https://companymap.co.uk/bot.html)07 · Verification
Is it really CompanyMapBot?
CompanyMapBot’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.
FAQ
Questions about CompanyMapBot
What is CompanyMapBot?
It is the crawler of CompanyMap. It links UK companies in the Companies House register to their websites.
Does CompanyMapBot respect robots.txt?
Yes. It checks robots.txt before every site and honours it, including User-agent: * rules. Its token is CompanyMapBot.
How can I remove my domain from CompanyMap?
Add a Disallow rule for CompanyMapBot, or email crawler@companymap.co.uk to request exclusion or removal.
Last updated Oct 19, 2026