Robots.txt Generator

Create and customize your robots.txt file to control how search engines crawl your website. Includes sitemap directives and best practices.

Configuration

Quick Templates

Sitemap URL

The URL of your XML sitemap (required for search engines)

Additional Sitemaps (Optional)

User Agents & Rules

User-agent

Disallow PathsAllow Paths (Optional)

Crawl-delay (seconds)

Optional: Time in seconds between requests (not supported by all crawlers)

No robots.txt generated yet

Configure settings and click Generate