llms.txt Explained: A Map of Your Site for AI

Q: Is llms.txt an official standard?

It is an emerging community convention, not yet a formal standard ratified by a body like the W3C. But it is cheap to ship and costs nothing if a given engine ignores it.

Q: Where does llms.txt go?

At the site root, served at /llms.txt, the same way robots.txt is served at /robots.txt. Some sites also publish an expanded llms-full.txt with the full text of key pages.

Q: How is llms.txt different from robots.txt?

robots.txt tells crawlers what they may not access; llms.txt tells language models what is most worth reading. One restricts, the other recommends.

llms.txt is a plain-text file that gives AI answer engines a curated map of your best pages. Here's the format, why it helps AEO, and how to generate it.

llms.txt is a plain-text file at the root of your site that gives AI answer engines a clean, curated map of your most important content. Think of it as robots.txt's purpose - guidance for machines - but aimed at large language models instead of crawlers, and pointing them toward your best pages rather than away.

What it looks like

It lives at /llms.txt and is plain Markdown: a title, a short description of the site, then linked lists of your key pages grouped by section.

# Bird CMS

> A markdown CMS your AI agent edits directly.

## Docs
- [Quickstart](/docs/quickstart): from clone to a published page
- [Content model](/docs/content-model): how content lives on disk

## Blog
- [What is AEO?](/blog/what-is-answer-engine-optimization)

Why it helps AEO

Answer engines have limited budget to crawl and parse a site. llms.txt hands them a short, authoritative list of what matters, in a format that is trivial to read - no HTML to strip, no navigation to guess. It raises the odds that the pages you care about are the ones a model reads and cites. It pairs naturally with the rest of Answer Engine Optimization.

Do you have to write it by hand?

You shouldn't have to. A CMS that knows your published content can generate llms.txt automatically and keep it current as you publish. Bird CMS does exactly that - it emits /llms.txt and sitemap.xml from your content with no build step. See the content model for what it includes.

Frequently asked questions

Is llms.txt an official standard?

It is an emerging community convention, not (yet) a formal standard ratified by a body like the W3C. But it is cheap to ship and costs nothing if a given engine ignores it, so there is little downside to adopting it early.

Where does llms.txt go?

At the site root, served at /llms.txt, the same way robots.txt is served at /robots.txt. Some sites also publish an expanded llms-full.txt with the full text of key pages.

How is llms.txt different from robots.txt?

robots.txt tells crawlers what they may not access; llms.txt tells language models what is most worth reading. One restricts, the other recommends - they solve opposite halves of the same machine-guidance problem.

Does llms.txt guarantee my pages get cited?

No. It improves the odds by making your best content easy to find and parse, but citation still depends on relevance, authority and the quality of the page itself. Treat it as one signal among several, not a switch.

Do I need to update llms.txt every time I publish?

Not if your CMS generates it. Hand-maintained files drift out of date quickly; a generated llms.txt - like the one Bird CMS emits - stays in sync with your published content automatically.