Known bots · Anthropic
What is ClaudeBot?
Anthropic’s crawler for AI training. tidekeeper knows it by its User-Agent.
- Run by
- Anthropic
- Reads for
- AI training
- Known by
- “claudebot, anthropic-ai” in its User-Agent, and ClaudeBot in robots.txt
- Its addresses
- Anthropic publishes no list, so a read is known by its name alone: anyone can send it.
- Its readers come as
- Claude, the assistant: a reader who follows a link from it arrives with Claude as the source.
- Its owner’s page
- support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
Keep it out
User-agent: ClaudeBot Disallow: /
Or only part of the site: Disallow: /private/ under the same User-agent line.
On tidekeeper.eu
ClaudeBot read tidekeeper.eu 194 times in the last 30 days, as tidekeeper counts its own site, from its server.