Known bots · OpenAI
What is OAI-SearchBot?
OpenAI’s crawler for its AI search. tidekeeper knows it by its User-Agent, and checks each read against the addresses OpenAI publishes.
- Run by
- OpenAI
- Reads for
- AI search
- Known by
- “oai-searchbot” in its User-Agent, and OAI-SearchBot in robots.txt
- Its addresses
- OpenAI publishes them: openai.com/searchbot.json. tidekeeper checks every read against the list.
- Its readers come as
- ChatGPT, the assistant: a reader who follows a link from it arrives with ChatGPT as the source.
- Its owner’s page
- developers.openai.com/api/docs/bots
Keep it out
User-agent: OAI-SearchBot Disallow: /
Or only part of the site: Disallow: /private/ under the same User-agent line.
On tidekeeper.eu
OAI-SearchBot read tidekeeper.eu 25 times in the last 30 days, as tidekeeper counts its own site, from its server.