01 · Operator
Who operates Dataprovider.com
- Company
- Dataprovider.com
- Type
- Other bot
- Official docs
- dataprovider.com
02 · Behavior
What Dataprovider.com does
The spider aims to index over 400 million domains and structure the data for Dataprovider.com's customers. It checks robots.txt and meta tags before indexing. Dataprovider.com also offers an opt-out for site owners.
03 · Impact
Why Dataprovider.com matters for your site
If you allow it
- Respects robots.txt and meta tags
- Your site may appear in business web data
If you block it
- Data is resold commercially
- Brings no search visibility
04 · Allow
How to allow Dataprovider.com
robots.txt
User-agent: dataprovider
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "dataprovider")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: dataprovider
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "dataprovider") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} dataprovider [NC]
# RewriteRule .* - [F,L]05 · Block
How to block Dataprovider.com
Dataprovider.com follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: dataprovider
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "dataprovider")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: dataprovider
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "dataprovider") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} dataprovider [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for Dataprovider.com
The user agent published by the operator:
Mozilla/5.0 (compatible; Dataprovider/6.101; +https://www.dataprovider.com/)07 · Verification
Is it really Dataprovider.com?
Anyone can copy a user agent. A reverse DNS lookup of a real request resolves to dataproviderbot.com, and that hostname resolves back to the same IP.
FAQ
Questions about Dataprovider.com
Does Dataprovider.com respect robots.txt?
Yes. Dataprovider.com states its spider follows robots.txt and META tags before indexing.
How do I block Dataprovider.com?
Add 'User-agent: dataprovider' with 'Disallow: /' to robots.txt. Dataprovider.com also offers an opt-out.
What does Dataprovider.com do with the data?
It structures data about websites into a database that it offers to its customers.
Last reviewed Oct 7, 2026