- [ Home ](/) / - Benefits & limitations # Benefits and limitations A frank look at what llms.txt does, what it does not, and how to decide whether to publish one. Last updated: September 8, 2026 On this page - [ Benefits ](#benefits) - [ Limitations ](#limitations) - [ Documented skepticism ](#skepticism) - [ When it is worth it ](#when) - [ When it is not ](#when-not) - [ How to measure impact ](#measure) ## Benefits ### 1. A shipped, machine-readable contract The first concrete benefit is internal: writing an `llms.txt` forces your team to agree on which pages actually represent your project. Most teams discover that the file they ship is shorter and clearer than their nav, and that exercise alone is valuable. ### 2. Better grounding for assistants that do read it A compatible assistant or custom retrieval workflow can be configured to fetch `llms.txt` and follow its links. A curated file then provides selected pages for that workflow. Publishing the file alone does not establish that Claude, ChatGPT, Perplexity, or another product fetches it automatically or uses it in answers; verify the behavior of the specific integration. ### 3. A stable, citable corpus With `llms-full.txt` alongside, an assistant can ingest your documentation as a single artifact and cite specific URLs back to the user. That citation behavior matters for trust and for click-through. ### 4. Disambiguation for ambiguous brands If your name collides with another company, product, or term, the blockquote summary at the top of `llms.txt` is a one-shot disambiguator. You control the first sentence an assistant reads about you. ### 5. A defensible baseline for "GEO" / "AEO" work Generative Engine Optimization (GEO) and Answer Engine Optimization (AEO) are still emergent disciplines. Publishing `llms.txt` creates a concrete resource for compatible clients, but it does not by itself prove or cause citations. ## Limitations ### 1. No major search engine confirms reading it Google Search explicitly says it does not use `llms.txt`. No major LLM provider has documented automatic inference-time use as a grounding signal. Publishing a file is not the same as confirming consumption on the receiving end. ### 2. It does not control crawler behavior `llms.txt` has no allow / disallow semantics. If you want to block `GPTBot` or `ClaudeBot`, you still need `robots.txt`. The two files solve different problems. ### 3. Adoption on the receiving side is uneven A compatible client or retrieval workflow can be configured to fetch `llms.txt`. Provider support must be verified product by product and must not be inferred from publication. ### 4. Spec is community-maintained, not standardized The proposal lives at [llmstxt.org](https://llmstxt.org/) and has not been through an IETF or W3C process. Expect minor changes, and validate against the live spec rather than against a frozen copy. ### 5. Easy to over-optimize The format invites the same sins as meta tags did in 2010: keyword stuffing, marketing copy, hidden agendas. Resist. The file is read by humans too, and reads as low-quality very quickly. ## Documented skepticism Healthy debate exists. The most-cited skeptical voice is John Mueller (Google Search Advocate), who has questioned both adoption and impact in multiple posts on Bluesky and Mastodon throughout 2025 and 2026. His summary, paraphrased: *publishing the file is cheap; expecting Google to consume it is wishful thinking.* The practical counter-argument is narrower: a site can justify the file when it has a compatible client or retrieval workflow to test and can keep the index current. This does not make `llms.txt` a universal grounding mechanism. Decide based on a named, measurable workflow rather than assumed provider behavior. Note Honest summary If you publish, do it because LLM assistants matter to your users, not because you expect Google rankings to move. Measure publication, retrieval, and use as separate outcomes. ## When it is worth it - **Documentation sites.** Highest ROI: assistants are already a primary discovery channel for developers. - **Developer tools and APIs.** Same reason. Pair with `llms-full.txt`. - **SaaS with technical buyers.** Buyers research with AI tools; a clean file improves what those tools say about you. - **Knowledge bases and reference sites.** Wikis, glossaries, and taxonomies benefit from explicit curation. - **Brands with name collisions.** The blockquote summary is your one shot at disambiguation. ## When it is probably not - **Pure e-commerce catalogs.** Product listings change too often; structured data (Schema.org) is a better bet. - **Visual or media-first sites.** Galleries, video platforms, and design portfolios get little from a Markdown text file. - **Sites without stable URLs.** If your URLs churn, your file rots, and stale URLs poison its reputation. - **Sites under heavy auth.** Public listings of private pages are useless and risky. ## How to measure impact Honest answer: measurement is hard. There is no Google Search Console for LLM citations. A practical instrument set: - **Server logs.** Filter for known LLM user-agents (`GPTBot`, `ClaudeBot`, `PerplexityBot`, `OAI-SearchBot`) hitting `/llms.txt` and `/llms-full.txt`. - **Referrer analysis.** Watch referrers from `chat.openai.com`, `claude.ai`, `perplexity.ai`. Volume there grew substantially across 2025. - **Manual prompts.** Periodically ask Claude, ChatGPT, and Perplexity about your product. Note whether they cite your domain and which URLs they reach for. - **Brand monitoring.** Tools like Profound, Otterly, Xfunnel, and AthenaHQ track LLM mentions of brands; they are early but improving fast. ## Next - [Best practices](/best-practices/), the rules and the mistakes. - [Real-world examples](/examples/). - [Validator](/validator/). ## Sources - [ llmstxt.org, community proposal ](https://llmstxt.org/) - [ John Mueller (Google), Bluesky thread on llms.txt ](https://bsky.app/profile/johnmu.com) - [ Search Engine Land, llms.txt is a treasure map for AI ](https://searchengineland.com/llms-txt-isnt-robots-txt-its-a-treasure-map-for-ai-456586) - [ SE Ranking, llms.txt adoption study (300k domains) ](https://seranking.com/blog/llms-txt/)