OpenAI’s bots

OpenAI runs 3 bots tidekeeper knows by name, each with its own name in robots.txt.

Its bots

OpenAI’s own page on them: developers.openai.com/api/docs/bots

Keep one out

Each reads by the group that names it. To keep GPTBot out and let the others read:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
User-agent: ChatGPT-User
Allow: /

ChatGPT-User opens a page because someone asked ChatGPT to, now. OpenAI says of it: “Because these actions are initiated by a user, robots.txt rules may not apply.”