01 · Operator
Who operates ev-crawler
- Company
- Headline
- Type
- Other bot
- Official docs
- headline.com
02 · Behavior
What ev-crawler does
It visits websites for public data such as home page content, job postings, team pages and locations. Headline uses this data to discover interesting companies. Site owners can ask to be excluded by email.
03 · Impact
Why ev-crawler matters for your site
If you allow it
- Can make your company visible to an investor
- Reads only public pages
If you block it
- No search or traffic benefit
- Commercial data collection
04 · Allow
How to allow ev-crawler
robots.txt
User-agent: ev-crawler
Allow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "ev-crawler")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: ev-crawler
Allow: /nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "ev\-crawler") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} ev\-crawler [NC]
# RewriteRule .* - [F,L]05 · Block
How to block ev-crawler
Start with robots.txt. If ev-crawler keeps showing up in your logs, block it at your CDN or web server.
robots.txt
User-agent: ev-crawler
Disallow: /Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "ev-crawler")
Action: BlockWordPress
# WordPress serves a virtual robots.txt. Edit it with your SEO plugin:
# Yoast: SEO › Tools › File editor · Rank Math: General Settings › Edit robots.txt
User-agent: ev-crawler
Disallow: /nginx
# In your server { } block:
if ($http_user_agent ~* "ev\-crawler") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ev\-crawler [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for ev-crawler
The user agent published by the operator:
Mozilla/5.0 (compatible; ev-crawler/1.0; +https://headline.com/web-crawler)07 · Verification
Is it really ev-crawler?
ev-crawler’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.
FAQ
Questions about ev-crawler
Who operates ev-crawler?
Headline, a venture capital firm. Its user agent links to headline.com/web-crawler.
What does ev-crawler collect?
Public information such as home page content, job postings, team pages and location references.
How do I stop ev-crawler?
Headline asks site owners to email crawler@headline.com to opt out. You can also add a robots.txt rule.
Last reviewed Oct 7, 2026