- [ Home ](/) / - Best practices # Best practices Ten rules, the mistakes we see most often, and concrete patterns for i18n, security, and CI. Last updated: August 12, 2026 ## Ten rules ℹ Version 2, published 10 August 2026 The proposal now supports files at any path, page-level Markdown alternates and the describedby relation. Use the most specific applicable file for a page. These are agent-discovery conventions, not Google ranking signals. - **Curate.** A short list of high-signal pages beats a long list of mediocre ones. Ten to thirty links can be a useful starting heuristic, but the intended tasks should determine the count. - **Use absolute URLs.** Always `https://yourdomain.com/...`. Relative URLs are technically allowed but fragile. - **Group by product surface.** Sections like *Product*, *Pricing*, *Developers* reflect how a user (and an LLM) thinks. Avoid blog/doc/guide buckets unless they map onto your real navigation. - **Keep the summary factual.** The blockquote after the H1 should read like a Wikipedia opener, not like a landing-page hero. - **One sentence per item.** The colon-prefixed note is for disambiguation, not for marketing. - **Use the `Optional` label sparingly.** It is a clear editorial home for press, brand assets and archives, but v2 gives it no special machine semantics. - **Mirror your stable URLs.** If a page in `llms.txt` moves, update or redirect it. Stale URLs poison the file’s reputation. - **Publish `llms-full.txt` only for a defined ingestion need.** Consolidated documentation can help a known consumer, but it also increases size, freshness and security costs. - **Run the [validator](/validator/) in CI.** A content migration that breaks your file should fail the build. - **Date your file.** A short note like *“Last reviewed 2026-04-01”* in the body is helpful for both humans and crawlers. ## Common mistakes - **No H1.** The H1 is the only required element. Without it, the file is invalid. - **Multiple H1s.** Use H2s for sections. There must be exactly one H1. - **Custom front matter.** YAML and JSON headers sit outside the published grammar and can make conforming parsers reject or misread the file. - **Pasted Markdown tables or images.** Keep the preamble focused and use link lists inside H2 sections. Extra structures add parsing ambiguity and often duplicate linked pages. - **Including auth-gated URLs.** If a page requires login, do not list it, the LLM will hit a wall. - **Overlong descriptions.** “The world’s most advanced AI-powered platform for next-gen synergistic transformation” helps no one. Keep each note only as long as needed to distinguish the destination. - **Listing 500 URLs without scope.** Recheck the tasks, split real content boundaries into path-level files, or provide a separate full-content resource for a known consumer. - **Blocking the resource unintentionally.** Make sure each declared root or path-level file is reachable under the crawl policy you intend. ⚠ Anti-pattern: using llms.txt for SEO injection Stuffing the file with keyword-rich notes does not help. There is no evidence that any major LLM extracts keyword density from llms.txt . It also reads as low-quality to anyone who fetches the file directly. ## Multilingual sites The proposal does not prescribe one internationalization architecture. Two design choices are common: - **One default-language file at the root.** The simplest option when the listed resources and intended consumers share a language. - **Per-locale variants.** Serve `/llms.txt` (default), `/fr/llms.txt`, `/es/llms.txt`. Link to them from your root file’s body or under an *Optional* section, or declare the applicable file with `rel="describedby"` on localized pages. Whichever pattern you pick, do not duplicate URL sets across locales: each variant should point to the localized version of each page. ## Security and privacy - **Everything in `llms.txt` is public.** Treat the file as broadcast. - **Never list staging or preview URLs.** Any client that retrieves the public file can see them. - **Do not list URLs with secrets in query strings.** This sounds obvious; we have seen it happen. - **If the page exposes user data behind auth, it does not belong here.** - **Audit the file at every release.** A leaked draft URL is the most common security mistake. Treat the file like external configuration, without assuming agents obey it. In the Ahrefs analysis of 137,210 domains, the largest user-agent string in the research category was `prompt-injection-survey/1.0`. That label proves neither an attack nor its operator, but it is a useful reminder to keep content factual, review changes and constrain any consuming agent. ## Automation in CI Treat `llms.txt` like any other artifact: generate, validate, and gate releases on it. - Generate it from your content source (CMS, MDX collection, database). - Run the [validator](/validator/) in CI; fail the build on any error. - Diff the file across releases; alert the docs owner on large deletions. - Smoke-test the production URL after deploy: `curl -fsS https://yourdomain.com/llms.txt | head -1`. ## Next - [Benefits and limitations](/benefits-limitations/), what to expect, what not to. - [Real-world examples](/examples/), copy what works. - [Validator](/validator/). ## Sources - [ llmstxt.org, official specification ](https://llmstxt.org/) - [ Ahrefs: We Analyzed 137K Sites, 97% of llms.txt Files Never Get Read (June 2026) ](https://ahrefs.com/blog/llmstxt-study/) On this page - [ Ten rules ](#rules) - [ Common mistakes ](#mistakes) - [ Multilingual sites ](#i18n) - [ Security and privacy ](#security) - [ Automation in CI ](#automation)