01 · Operator
Who operates SISTRIX Crawler
- Company
- SISTRIX GmbH
- Type
- SEO tool crawler
- Official docs
- sistrix.com
02 · Behavior
What SISTRIX Crawler does
The crawler powers the SISTRIX Optimizer, which runs on-page analysis and website audits. It reads robots.txt first and uses listed sitemaps as crawl sources. It does not execute JavaScript or store cookies. It adjusts its crawl rate to the page loading speed.
03 · Impact
Why SISTRIX Crawler matters for your site
If you allow it
- You or your agency use SISTRIX Optimizer audits
- It follows robots.txt and adapts its crawl rate
If you block it
- You do not use SISTRIX
- Audits by third parties give you no direct benefit
04 · Allow
How to allow SISTRIX Crawler
robots.txt
User-agent: sistrix
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "sistrix")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: sistrix
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "sistrix") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} sistrix [NC]
# RewriteRule .* - [F,L]05 · Block
How to block SISTRIX Crawler
SISTRIX Crawler follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.
robots.txt
User-agent: sistrix
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "sistrix")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: sistrix
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "sistrix") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} sistrix [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for SISTRIX Crawler
The user agent published by the operator:
Mozilla/5.0 (compatible; SISTRIX Crawler; http://crawler.sistrix.net/)07 · Verification
Is it really SISTRIX Crawler?
Anyone can copy a user agent. A reverse DNS lookup of a real request resolves to crawler.sistrix.net, and that hostname resolves back to the same IP.
FAQ
Questions about SISTRIX Crawler
Does the SISTRIX Crawler respect robots.txt?
Yes. It reads robots.txt before crawling and follows rules for User-agent: sistrix and User-agent: *.
How can I verify the SISTRIX Crawler?
Its IP addresses resolve to hostnames under crawler.sistrix.net, for example 136-243-92-8.crawler.sistrix.net. The crawler runs from IP addresses in Germany.
Does the SISTRIX Crawler render JavaScript?
No. SISTRIX states that its crawler does not use JavaScript, so pages must be readable as static HTML.
Last reviewed Oct 8, 2026