20 bots tracked

Bot directory

Every crawler that visits websites, from AI training bots to search engines: who runs it, what it does, and how to allow or block it. Live activity data from Log Hero.

Search bots

AhrefsBot Ahrefs

SEO tool crawler Your call – –
Amazonbot Amazon

AI training crawler Your call – –
Applebot Apple

Search engine crawler Allow – –
Applebot-Extended Apple

AI training crawler Your call – –
Bingbot Microsoft

Search engine crawler Allow – –
Bytespider ByteDance

AI training crawler Block – –
CCBot Common Crawl

AI training crawler Your call – –
ChatGPT-User OpenAI

AI assistant fetcher Allow – –
Claude-SearchBot Anthropic

AI search crawler Allow – –
Claude-User Anthropic

AI assistant fetcher Allow – –
ClaudeBot Anthropic

AI training crawler Your call – –
DuckAssistBot DuckDuckGo

AI search crawler Allow – –
Google-Extended Google

AI training crawler Your call – –
Googlebot Google

Search engine crawler Allow – –
GPTBot OpenAI

AI training crawler Your call – –
Meta-ExternalAgent Meta

AI training crawler Your call – –
OAI-SearchBot OpenAI

AI search crawler Allow – –
Perplexity-User Perplexity

AI assistant fetcher Allow – –
PerplexityBot Perplexity

AI search crawler Allow – –
SemrushBot Semrush

SEO tool crawler Your call – –

Your site

See which bots crawl your website

Log Hero reads your server logs and shows every AI bot, every page, every day.

Free during early access