Robots.txt Generator – Free Online Tool | Trencada
A robots.txt file tells search engine crawlers which parts of your site they may and may not crawl. Getting the syntax wrong can accidentally block your whole site from search results — or leave staging areas exposed. Trencada's Robots.txt Generator builds a valid robots.txt file for you in seconds: pick a preset, fine-tune crawl rules for any bot, add your sitemap, then copy or download the file. It runs 100% in your browser, is completely free, and needs no signup.
How to Use the Robots.txt Generator
- Start with a quick preset. Click Allow all, Block all, WordPress, or Googlebot only to load a ready-made rule set.
- Set the user-agent. Each crawl rule group targets one bot — use
* for all bots or a name like Googlebot. Add more groups with "+ Add user-agent group".
- Add Allow and Disallow rules. For each rule, choose Allow or Disallow from the dropdown and enter a path such as
/wp-admin/. An empty Disallow path means "allow everything". Use "+ Allow rule" and "+ Disallow rule" to add more.
- Add your sitemap (optional). Paste your sitemap URL, e.g.
https://example.com/sitemap.xml, so crawlers can discover all your pages quickly.
- Review the generated file. The robots.txt output updates live as you type. The tool warns you if a path doesn't start with
/.
- Copy or download. Copy the file to your clipboard or download it as
robots.txt, then upload it to your domain's root (e.g. example.com/robots.txt). Use Reset to start over.
Key Features
- 4 one-click presets: Allow all, Block all, WordPress, and Googlebot only.
- Multiple user-agent groups — set different rules for different bots.
- Unlimited Allow/Disallow rules per group, with live syntax validation.
- Sitemap directive to point crawlers at your XML sitemap.
- Live preview of the generated robots.txt as you edit.
- Copy to clipboard or download the finished file as
robots.txt.
- 100% client-side — your rules never leave your browser.
Why Choose Trencada?
- Free and unlimited — generate as many robots.txt files as you need.
- No signup, no uploads — everything runs locally in your browser.
- Built-in guidance — the tool's "How it works" notes explain rule order, empty Disallow lines, and why robots.txt is a public hint, not security.
- Part of a complete web toolkit — if you disallow sensitive areas, protect them properly too with our htpasswd generator, and handle on-page basics with our meta tag generator.
Frequently Asked Questions
What is a robots.txt file?
A robots.txt file is a plain-text file placed at your website's root that tells search engine crawlers (like Googlebot) which pages or sections they are allowed to crawl. Most legitimate crawlers respect these rules voluntarily.
Where do I put the robots.txt file?
Upload the downloaded file to the root of your domain, so it is reachable at https://example.com/robots.txt. That is the only location crawlers check.
Does robots.txt stop pages from appearing in Google?
Not reliably. Robots.txt controls crawling, not indexing or access. If a page must stay private, protect it with a password (for example with our htpasswd generator) or use a noindex directive — don't rely on robots.txt alone.
What is the difference between Allow and Disallow?
Disallow: /path/ asks crawlers to skip that path, while Allow: /path/ explicitly permits it. Crawlers use the most specific matching rule first when both could apply.
Is this robots.txt generator free?
Yes. The Robots.txt Generator on Trencada is completely free — no signup, no limits, and everything runs in your browser.