AI Crawl Checker
See who you're letting in
Paste your robots.txt and this tool shows, bot by bot, which AI crawlers you allow, block, or never mention. If GPTBot, ClaudeBot, or PerplexityBot can't reach your pages, you can't be cited by the engines they feed. The crawl checks run in your browser. An optional AI fix uses your OpenRouter key when provided, or our shared free service otherwise.
Your robots.txt
No API key is required for the free tier. Bring your own OpenRouter key in AI settings to use more capable paid models.
Results
Paste your robots.txt and press Check bots.