Oh Dear Crawler User Agent: What It Is & How to Block It

01 · Operator

Who operates Oh Dear

Company: Oh Dear
Type: Monitoring bot
Official docs: ohdear.app

02 · Behavior

What Oh Dear does

The crawler visits sites monitored by Oh Dear customers. It also checks your pages when a customer monitors outgoing links to your site. It throttles itself and respects robots.txt by default. Current user agents start with OhDear.app; older versions used OhDear/1.1.

03 · Impact

Why Oh Dear matters for your site

If you allow it

Needed if you or a partner uses Oh Dear monitoring
Respects robots.txt and throttles requests

If you block it

No benefit if nobody monitors your site with Oh Dear
Adds crawl load from link checks

04 · Allow

How to allow Oh Dear

robots.txt

User-agent: OhDear
Allow: /

Cloudflare

# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "OhDear")
Action:     Skip › All Super Bot Fight Mode rules

# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.

WordPress

# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: OhDear
Allow: /

nginx

# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "OhDear") { return 403; }

Apache

# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} OhDear [NC]
# RewriteRule .* - [F,L]

05 · Block

How to block Oh Dear

Oh Dear follows robots.txt, so one rule is enough. Use a server or CDN rule only to stop spoofed copies.

robots.txt

User-agent: OhDear
Disallow: /

Cloudflare

# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "OhDear")
Action:     Block

WordPress

# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: OhDear
Disallow: /

nginx

# In your server { } block:
if ($http_user_agent ~* "OhDear") {
    return 403;
}

Apache

# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} OhDear [NC]
RewriteRule .* - [F,L]
</IfModule>

06 · User agents

User agents we see for Oh Dear

The user agent published by the operator:

OhDear.app (+https://ohdear.app/docs/features/broken-links-detection)

07 · Verification

Is it really Oh Dear?

Oh Dear’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.

FAQ

Questions about Oh Dear

Why is Oh Dear crawling my site?

Either your site is monitored by Oh Dear, or an Oh Dear customer checks outgoing links that point to your site.

Does Oh Dear respect robots.txt?

Yes. Oh Dear says its crawler throttles itself and respects robots.txt by default.

What user agent does Oh Dear use?

Its user agents start with OhDear.app, followed by a link to the feature that triggered the check.

Oh Dear is Oh Dear’s monitoring bot