Go-http-client
The default UA of Go's standard HTTP client — common in monitoring agents, custom crawlers, and bulk fetchers.
| robots.txt token | Go-http-client |
| Respects robots.txt | No — unless the program adds it |
| Verification | None — a default the developer never changed |
| Our recommendation | Block — Same as every default tool UA: legitimate automation will identify itself; this didn't. |
Block Go-http-client with robots.txt
User-agent: Go-http-client
Disallow: /robots.txt is a request, not a wall — requests claiming to be Go-http-client from unverified networks should be treated as bad bots. How to verify crawlers by network →
See every Go-http-client request hitting your site — live, with per-bot allow / block / tarpit controls. Try TrafficDATA free (100K page views)