Known bots · Anthropic

Anthropic’s bots

Anthropic runs 3 bots tidekeeper knows by name, each with its own name in robots.txt.

Its bots

ClaudeBotAI trainingby its nameread us 236× this monthClaude-SearchBotAI searchby its namenot seen here this monthClaude-UserFor someone nowby its namenot seen here this month

Anthropic’s own page on them: support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler

Keep one out

Each reads by the group that names it. To keep ClaudeBot out and let the others read:

robots.txt
User-agent: ClaudeBot
Disallow: /

User-agent: Claude-SearchBot
User-agent: Claude-User
Allow: /
See which of them read your site: Bots and AI crawlers →