HeadlessChrome
Real Chrome running without a window — the engine under Puppeteer and Playwright. It executes JavaScript and looks like a browser because it is one; the UA token is the honest default that automation forgot to remove.
| robots.txt token | HeadlessChrome |
| Respects robots.txt | No — it's a browser, not a crawler; the controlling script decides |
| Verification | TLS and protocol genuinely match Chrome — behavior, pacing, and network reputation are the tells |
| Our recommendation | Block — Legitimate headless use (your own monitoring, previews) is traffic you recognize; everyone else's is scraping. |
Block HeadlessChrome with robots.txt
User-agent: HeadlessChrome
Disallow: /robots.txt is a request, not a wall — requests claiming to be HeadlessChrome from unverified networks should be treated as bad bots. How to verify crawlers by network →
See every HeadlessChrome request hitting your site — live, with per-bot allow / block / tarpit controls. Try TrafficDATA free (100K page views)