Known bots · Meta

Meta’s bots

Meta runs 5 bots tidekeeper knows by name, each with its own name in robots.txt.

Its bots

Meta-ExternalAgentAI trainingby its namenot seen here this monthMeta-ExternalFetcherFor someone nowby its namenot seen here this monthMeta-WebIndexerAI searchby its namenot seen here this monthfacebookexternalhitLink previewby its nameread us 18× this monthWhatsAppLink previewby its nameread us 1× this month

Meta’s own page on them: developers.facebook.com/docs/sharing/webmasters/web-crawlers

Keep one out

Each reads by the group that names it. To keep Meta-ExternalAgent out and let the others read:

robots.txt
User-agent: Meta-ExternalAgent
Disallow: /

User-agent: Meta-ExternalFetcher
User-agent: Meta-WebIndexer
User-agent: facebookexternalhit
User-agent: WhatsApp
Allow: /
See which of them read your site: Bots and AI crawlers →