Meta-ExternalFetcher
Fetches individual links on direct user request inside Meta products.
| robots.txt token | Meta-ExternalFetcher |
| Respects robots.txt | May bypass robots.txt for explicit user-requested fetches |
| Verification | Documented by Meta |
| Our recommendation | Allow — Link unfurling and user fetches — blocking breaks previews more than it stops scraping. |
Block Meta-ExternalFetcher with robots.txt
User-agent: Meta-ExternalFetcher
Disallow: /robots.txt is a request, not a wall — requests claiming to be Meta-ExternalFetcher from unverified networks should be treated as bad bots. How to verify crawlers by network →
See every Meta-ExternalFetcher request hitting your site — live, with per-bot allow / block / tarpit controls. Try TrafficDATA free (100K page views)