Free tool · AI crawlers

Can AI read your site?

Give a domain: tidekeeper reads its robots.txt as each AI crawler would, looks for an llms.txt, and fetches the page the way a crawler that runs no script gets it.

robots.txt, llms.txt and the page: three requests, nothing kept.

Then, which of them actually came

robots.txt says who may read. tidekeeper's Bots and AI says who did: each crawler by name, the pages it read, whether its address was its owner's, and the readers each assistant sent after.

Bots and AI crawlers →