llms.txt

llms.txt is a proposed standard — a Markdown-formatted file at a site’s root (/llms.txt) — that curates and points AI systems to a site’s most important content in a clean, easy-to-parse form. It is conceptually a companion to robots.txt and sitemaps, but aimed at large language models rather than traditional crawlers.

How it works and where it stands

The file lists key pages with short descriptions, and often links to Markdown versions of them, so an LLM can find and ingest the most relevant, distilled content without wading through navigation, scripts, and boilerplate. It is an emerging proposal, not an official standard, and adoption by major AI providers is still limited and evolving, so treat it as a low-cost, forward-looking experiment rather than a guaranteed ranking factor. The durable fundamentals — clean HTML, structured data, and answer-first content — matter more today, but publishing an llms.txt is inexpensive insurance as the ecosystem forms.

Where it applies

• Documentation and developer sites — early adopters, where distilled, LLM-ready reference content is valuable.
• Content-heavy brands — curating the pages you most want AI systems to draw on.
• AEO experimentation — a low-risk tactic to test as standards mature.

What matters from each seat

• In-house — publish it as an experiment, but don’t divert effort from the fundamentals that demonstrably work.
• Engineering — generate it from the same source as your sitemap so it stays current automatically.
• Agency and consulting — frame it honestly to clients: promising and cheap, not yet proven.