NiceCrawler User Agent: What It Is & How to Block It

01 · Operator

Who operates NiceCrawler

Company: Intelium Corp.
Type: Other bot

02 · Behavior

What NiceCrawler does

NiceCrawler identifies itself with a link to nicecrawler.com. Intelium builds software for the domain name and cybersecurity industries and analyzes internet data for clients. The crawler's exact use is not publicly documented.

03 · Impact

Why NiceCrawler matters for your site

If you allow it

Identifies itself with a link
Supports domain and security data services

If you block it

No direct search traffic
Purpose not documented by the operator

04 · Allow

How to allow NiceCrawler

robots.txt

User-agent: Nicecrawler
Allow: /

Cloudflare

# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "Nicecrawler")
Action:     Skip › All Super Bot Fight Mode rules

# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.

WordPress

# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: Nicecrawler
Allow: /

nginx

# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "Nicecrawler") { return 403; }

Apache

# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} Nicecrawler [NC]
# RewriteRule .* - [F,L]

05 · Block

How to block NiceCrawler

Start with robots.txt. If NiceCrawler keeps showing up in your logs, block it at your CDN or web server.

robots.txt

User-agent: Nicecrawler
Disallow: /

Cloudflare

# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "Nicecrawler")
Action:     Block

WordPress

# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: Nicecrawler
Disallow: /

nginx

# In your server { } block:
if ($http_user_agent ~* "Nicecrawler") {
    return 403;
}

Apache

# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Nicecrawler [NC]
RewriteRule .* - [F,L]
</IfModule>

06 · User agents

User agents we see for NiceCrawler

The user agent published by the operator:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Nicecrawler/1.1; +http://www.nicecrawler.com/) Chrome/90.0.4430.97 Safari/537.36

07 · Verification

Is it really NiceCrawler?

NiceCrawler’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.

FAQ

Questions about NiceCrawler

Who operates NiceCrawler?

Matomo's bot database lists Intelium Corp., a US software company for the domain and cybersecurity industries.

Should I block NiceCrawler?

It is optional. It brings no search traffic, and its robots.txt behavior is not documented.

NiceCrawler is Intelium Corp.’s other bot