robots.txt Generator

Allow the right bots, on purpose

A correct robots.txt is the front door to AI citation: if GPTBot, ClaudeBot, or PerplexityBot can't crawl you, you can't be quoted. Choose which AI crawlers to allow or block, add your sitemap, and this tool writes a clean, explicit robots.txt you can copy or download. It runs entirely in your browser.