What is llms.txt?

A concise definition of the proposed llms.txt convention, its current scope, measured adoption, and limits.

Last updated: August 25, 2026

Definition

llms.txt is a Markdown file published at a website root or a specific path. It gives compatible agents a short, curated map of useful resources under that scope. The familiar root URL is https://example.com/llms.txt.

It was proposed by Jeremy Howard at Answer.AI in September 2024 as a community convention, not a formal standard. The canonical specification lives at llmstxt.org.

llms.txt, llmstxt, llms txt, or llmtxt?

The proposed filename is llms.txt. People also search for llmstxt, llms txt, llmtxt, and llm.txt when they omit punctuation, misremember the plural, or refer to the topic rather than the literal file.

Those spellings do not name separate formats. Use llms.txt in documentation and at the published URL. llmtxt.info is this reference site; llmstxt.org publishes the current v2 proposal. This page groups the variants instead of creating duplicate pages.

Origin

Howard’s original argument: large language models have small context windows compared to a typical website. Asking a model to crawl an entire site every time someone wants to ask a question about your product is wasteful and lossy. Sites already publish robots.txt for crawlers and sitemap.xml for search engines, llms.txt proposes a parallel convention specifically for LLMs.

The format is intentionally minimal:

Markdown (so a human can read and edit it).
Predictable structure (so a parser can reliably extract sections and links).
Curated, not exhaustive (the goal is signal, not coverage).

What problem it solves

Context window limits. Even with multi-million-token models, loading a full documentation site costs latency and money. A focused llms.txt can reduce unnecessary retrieval when a compatible client needs only a small set of canonical resources.
Discovery noise. A site’s most valuable content for an LLM is rarely the same as its most visited content. llms.txt lets you tell models explicitly: “these are the pages worth reading.”
Reviewable contracts. When the file is version-controlled and checked in CI, its URLs become an explicit surface that can be tested when pages move.
Companion file llms-full.txt. A sibling convention used by several documentation publishers inlines the content of linked pages as concatenated Markdown, so an LLM client can load a single URL and get the full, ready-to-feed corpus.

What llms.txt is not

Plenty of confusion exists in the wild. To be clear:

It is not a W3C, IETF, or otherwise standardized protocol. It is a community proposal.
It is not enforced by any major LLM provider. No vendor (OpenAI, Anthropic, Google, Meta) has publicly committed to consistently fetching it. Adoption today is mostly opportunistic.
It is not a ranking signal. Publishing llms.txt does not make your site rank higher in classic search nor in AI Overviews. There is no public evidence to that effect as of August 2026.
It does not replace robots.txt, sitemap.xml, or schema.org. See the dedicated comparison page.
It is not a security mechanism. Anything you list in llms.txt is, by definition, public.

Adoption status (2026)

Adoption is real but uneven. Most early adopters are documentation platforms and developer-tools companies: Anthropic, Cloudflare, Vercel, Stripe, Mintlify, Perplexity. Outside the dev-tools world, coverage is still thin.

Our fixed technology-oriented panel found 119 valid root files among 217 reachable hosts on 2026-08-31, a rate of 54.8%. This is a transparent panel measurement, not a Web-wide adoption estimate.

Skepticism is also documented. Google’s John Mueller has been publicly cautious, and Mintlify themselves have acknowledged the skepticism head-on: the file is useful as a stable, machine-readable contract, but its real-world consumption by LLMs depends on each vendor’s evolving choices.

Typical use cases

API documentation. Map an LLM directly to your endpoint reference pages and tutorials.
SaaS marketing site. Help an assistant answer questions about your product, pricing, and integrations from a small set of canonical pages.
Open-source project. Point clients to the README, contribution guide, examples, and changelog.
Knowledge base. Surface the small, high-quality answers rather than every help-center article.

Quick FAQ

Is llms.txt required? No. It is voluntary.

Will it improve my SEO? Not directly. Read our benefits and limitations page for a careful answer.

How do I create one? See the step-by-step guide, or use the generator to scaffold a baseline.

Continue reading

The llms.txt file, locations, scope, and discovery.
Format reference, required and optional elements.
How llms.txt works, file structure and parsing rules.
Comparison with robots.txt and sitemap.xml.
Best practices, ten rules and common mistakes.