01 · Operator
Who operates BuiltWith
- Company
- BuiltWith
- Type
- Other bot
- Official docs
- builtwith.com
02 · Behavior
What BuiltWith does
It reads publicly visible code on websites to build technology profiles. These profiles power BuiltWith's lookup tool and lead lists. BuiltWith says it does not store page content. It checks robots.txt during indexing.
03 · Impact
Why BuiltWith matters for your site
If you allow it
- Keeps your public technology profile accurate
- Respects robots.txt
If you block it
- Profiles are sold as sales and market data
- No direct traffic benefit
04 · Allow
How to allow BuiltWith
robots.txt
User-agent: BuiltWith
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "BuiltWith")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: BuiltWith
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "BuiltWith") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} BuiltWith [NC]
# RewriteRule .* - [F,L]05 · Block
How to block BuiltWith
BuiltWith follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: BuiltWith
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "BuiltWith")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: BuiltWith
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "BuiltWith") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} BuiltWith [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for BuiltWith
The user agent published by the operator:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; BuiltWith/1.0; +http://builtwith.com/biup) Chrome/74.0.3729.131 Safari/537.3607 · Verification
Is it really BuiltWith?
BuiltWith’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.
FAQ
Questions about BuiltWith
What is the BuiltWith bot?
It is BuiltWith's crawler. It analyses public code to identify the technologies a website uses.
Does BuiltWith respect robots.txt?
Yes. BuiltWith says it checks robots.txt and stops if it finds a blocking rule.
How do I block BuiltWith?
Add 'User-agent: BuiltWith' with 'Disallow: /' to robots.txt. Site owners can also use builtwith.com/removals.
Last reviewed Oct 7, 2026