Google-Extended
Not a separate crawler — a robots.txt token that controls whether content Googlebot already crawls may also be used for Gemini training. Disallowing it does NOT affect your search indexing.
| robots.txt token | Google-Extended |
| Respects robots.txt | Yes — it exists only as a robots.txt control |
| Verification | Applies to regular Googlebot crawling (reverse DNS *.googlebot.com) |
| Our recommendation | Your call — Free to disallow without touching search rankings — the rare no-downside opt-out. |
Block Google-Extended with robots.txt
User-agent: Google-Extended
Disallow: /robots.txt is a request, not a wall — requests claiming to be Google-Extended from unverified networks should be treated as bad bots. How to verify crawlers by network →
See every Google-Extended request hitting your site — live, with per-bot allow / block / tarpit controls. Try TrafficDATA free (100K page views)