Claude-User
Fetches pages on demand when a Claude user asks about them.
| robots.txt token | Claude-User |
| Respects robots.txt | Yes — honors its own token |
| Verification | Documented by Anthropic |
| Our recommendation | Allow — User-initiated retrieval — blocking it mostly frustrates real people. |
Block Claude-User with robots.txt
User-agent: Claude-User
Disallow: /robots.txt is a request, not a wall — requests claiming to be Claude-User from unverified networks should be treated as bad bots. How to verify crawlers by network →
See every Claude-User request hitting your site — live, with per-bot allow / block / tarpit controls. Try TrafficDATA free (100K page views)