← All free tools
Robots.txt Generator
Generates a clean robots.txt with your sitemap and explicit choices for AI crawlers — GPTBot, ClaudeBot, PerplexityBot — which most generators still ignore.
Generates a clean robots.txt with your sitemap and explicit choices for AI crawlers — GPTBot, ClaudeBot, PerplexityBot — which most generators still ignore.