Known bots · OpenAI
What is GPTBot?
OpenAI’s crawler for AI training. tidekeeper knows it by its User-Agent, and checks each read against the addresses OpenAI publishes.
- Run by
- OpenAI
- Reads for
- AI training
- Known by
- “gptbot” in its User-Agent, and GPTBot in robots.txt
- Its addresses
- OpenAI publishes them: openai.com/gptbot.json. tidekeeper checks every read against the list.
- Its readers come as
- ChatGPT, the assistant: a reader who follows a link from it arrives with ChatGPT as the source.
- Its owner’s page
- developers.openai.com/api/docs/bots
Keep it out
User-agent: GPTBot Disallow: /
Or only part of the site: Disallow: /private/ under the same User-agent line.
On tidekeeper.eu
GPTBot read tidekeeper.eu 514 times in the last 30 days, as tidekeeper counts its own site, from its server.