01 · Operator
Who operates Yahoo! JAPAN Crawler
- Company
- Yahoo! JAPAN
- Official docs
- support.yahoo-net.jp
02 · Behavior
What Yahoo! JAPAN Crawler does
Yahoo! JAPAN lists the agents Y!J-BRU/VSIDX, Y!J-BRW, Y!J-ASR, Y!J-MMP/dscv and Y!J-WSC. They collect and store web content for search, research and development. The company warns that names and IPs can change without notice.
03 · Impact
Why Yahoo! JAPAN Crawler matters for your site
If you allow it
- Supports visibility in Yahoo! JAPAN services
- Relevant for sites with Japanese audiences
If you block it
- Little value for sites without Japanese visitors
- Names and IPs are not fixed, so hard to verify
04 · Allow
How to allow Yahoo! JAPAN Crawler
robots.txt
# Yahoo! JAPAN Crawler has no documented robots.txt token and does not read robots.txt.
# Use a Cloudflare, nginx or Apache rule instead.Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "Yahoo! JAPAN Crawler")
Action: Skip › All Super Bot Fight Mode rules
# Also check Security › Bots: "Block AI bots" can block it regardless of robots.txt.WordPress
# Yahoo! JAPAN Crawler has no documented robots.txt token and does not read robots.txt.
# Use a Cloudflare, nginx or Apache rule instead.nginx
# nginx serves every user agent by default.
# Make sure no rule like this blocks it:
# if ($http_user_agent ~* "Yahoo\! JAPAN Crawler") { return 403; }Apache
# Apache serves every user agent by default.
# Make sure .htaccess has no rule like this:
# RewriteCond %{HTTP_USER_AGENT} Yahoo\! JAPAN Crawler [NC]
# RewriteRule .* - [F,L]05 · Block
How to block Yahoo! JAPAN Crawler
Start with robots.txt. If Yahoo! JAPAN Crawler keeps showing up in your logs, block it at your CDN or web server.
robots.txt
# Yahoo! JAPAN Crawler has no documented robots.txt token and does not read robots.txt.
# Use a Cloudflare, nginx or Apache rule instead.Cloudflare
# Security › WAF › Custom rules › Create rule
Expression: (http.user_agent contains "Yahoo! JAPAN Crawler")
Action: BlockWordPress
# Yahoo! JAPAN Crawler has no documented robots.txt token and does not read robots.txt.
# Use a Cloudflare, nginx or Apache rule instead.nginx
# In your server { } block:
if ($http_user_agent ~* "Yahoo\! JAPAN Crawler") {
return 403;
}Apache
# .htaccess
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Yahoo\! JAPAN Crawler [NC]
RewriteRule .* - [F,L]
</IfModule>06 · User agents
User agents we see for Yahoo! JAPAN Crawler
The user agent published by the operator:
Mozilla/5.0 (compatible; Y!J-WSC/1.0; +https://yahoo.jp/3BSZgF)07 · Verification
Is it really Yahoo! JAPAN Crawler?
Yahoo! JAPAN Crawler’s operator publishes no IP ranges or hostnames, so requests cannot be verified. Treat the user agent as a claim, and watch your logs for unusual request rates.
FAQ
Questions about Yahoo! JAPAN Crawler
What does Y!J-WSC or Y!J-ASR mean in my logs?
These are crawlers of Yahoo! JAPAN. They collect web content for its search services and for research.
Why do some Yahoo Japan agents appear as YJ- without the "!"?
Yahoo! JAPAN states that agent names may omit the "!" mark, so both Y!J- and YJ- forms occur.
Last updated Oct 14, 2026