01 · Operator
Who operates ShapBot
- Company
- Parallel Web Systems
- Type
- AI search crawler
- Official docs
- parallel.ai
02 · Behavior
What ShapBot does
ShapBot crawls publicly available pages to build and maintain Parallel's search index. Parallel offers web search and web APIs built on this index. A separate agent, Shap-User, fetches pages on behalf of users and is not a crawler.
03 · Impact
Why ShapBot matters for your site
If you allow it
- Your pages can appear in Parallel's search features and web APIs
- Parallel publishes an IP list for verification
If you block it
- Content is indexed for use by a commercial search API
- Crawl volume can be high on large sites
04 · Allow
How to allow ShapBot
robots.txt
User-agent: ShapBot
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "ShapBot")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: ShapBot
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "ShapBot") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} ShapBot [NC]
# RewriteRule .* - [F,L]05 · Block
How to block ShapBot
ShapBot follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: ShapBot
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "ShapBot")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: ShapBot
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "ShapBot") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ShapBot [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for ShapBot
| User agent | Share | Last seen | Status |
|---|---|---|---|
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0 |
100% | Unverified |
07 · Verification
Is it really ShapBot?
Anyone can copy a user agent. Real ShapBot requests come from the IP ranges published at docs.parallel.ai/resources/shapbot.json.
FAQ
Questions about ShapBot
Who operates ShapBot?
ShapBot is operated by Parallel Web Systems. It builds and maintains Parallel's search index.
How do I block ShapBot?
Add a robots.txt group with User-agent: ShapBot and Disallow: /. Parallel names this token as the way to manage its crawler.
How can I verify that a request comes from ShapBot?
Compare the source IP with Parallel's published list at docs.parallel.ai/resources/shapbot.json. Requests claiming to be ShapBot from other IPs may be spoofed.
Last reviewed Oct 8, 2026