Does llms.txt work?

The short answer

It depends entirely on what you mean by "work."

If you want llms.txt to improve your Google rankings: no, it does not work for that. Google has publicly stated it does not use llms.txt as a ranking signal.

If you want llms.txt to help AI assistants (Claude, ChatGPT, Perplexity) understand and accurately cite your site: the early evidence is positive, but the receiving-side ecosystem is still fragmented. Agent frameworks (Cursor, Windsurf), RAG pipelines, and a growing number of MCP integrations actively look for it. The major LLM providers have not made a public commitment to fetch it automatically at inference time.

Cost of publishing one: a few hours. Upside: real-world improvements in how AI tools describe your product. For developer tools, documentation sites, and SaaS with technical buyers, that tradeoff is an easy yes.

What "working" depends on your goal

Before looking at evidence, it helps to be precise. People ask "does llms.txt work?" with four different goals in mind:

1. Goal A — LLM citation: When someone asks an AI assistant about my product, does it give accurate, up-to-date answers and cite my pages?
2. Goal B — AI crawler coverage: Do GPTBot, ClaudeBot, PerplexityBot fetch my llms.txt and use it to prioritize what they crawl?
3. Goal C — Google SEO: Does publishing llms.txt improve my ranking on Google?
4. Goal D — Developer tooling: Does Cursor, Windsurf, or a custom RAG pipeline pick up llms.txt to ground its answers?

Each goal has a different answer. We cover them in order below.

Does it help LLM citations? (Goal A)

The honest caveat first: there is no Google Search Console equivalent for LLM citations. You cannot open a dashboard and see "Perplexity fetched your llms.txt and cited you 47 times this week." The measurement problem is real.

That said, the mechanism is sound. llms.txt gives an AI system a curated reading list: instead of having to crawl and rank thousands of pages, the assistant gets a short Markdown file that says "these are the 15 pages that matter most for understanding this product." When an assistant uses that list as context before answering, it produces more accurate answers — and is more likely to cite the specific pages you pointed to.

Several agent frameworks — Cursor and Windsurf being the most widely used as of 2026 — have built explicit support for fetching llms.txt before loading project context. Their documentation confirms this behavior. For developer tools that reach these users, the citation benefit is concrete, not hypothetical.

The weakest part of Goal A is the major inference-time LLM providers (OpenAI, Anthropic, Google). None has publicly confirmed that ChatGPT, Claude, or Gemini fetches /llms.txt at inference time when a user asks a question. The file likely benefits retrieval-augmented pipelines more than it benefits base models responding from their training weights.

Verdict on Goal A: Works for agent frameworks and RAG pipelines. Unconfirmed for direct inference-time LLM use.

Do AI crawlers actually fetch it? (Goal B)

Yes — with a nuance. AI web crawlers (GPTBot from OpenAI, ClaudeBot from Anthropic, PerplexityBot, OAI-SearchBot) crawl the open web to build training corpora and search indexes. These bots respect robots.txt like any other crawler.

If your llms.txt is reachable at the root of your domain and not blocked by robots.txt, these crawlers will index it as they index any other page. Whether they treat it as a priority signal for crawling the rest of your site is a different question — none of the providers has published documentation confirming that behavior specifically for llms.txt.

What you can observe directly: check your server logs for requests to /llms.txt from known AI bot user-agents. Sites with llms.txt published routinely report hits from GPTBot and ClaudeBot within days of publishing. This confirms the file is being fetched; it does not prove it is being acted on structurally.

Verdict on Goal B: AI crawlers do fetch the file. Whether they use it to prioritize crawling is unconfirmed.

Does it help Google SEO? (Goal C)

No. This is the clearest answer of the four.

John Mueller, Search Relations lead at Google, addressed this directly in early 2025. His position is that llms.txt does not function as a ranking signal for Google Search. Google uses its own crawling and indexing infrastructure; it does not defer to a site-provided curation file as a substitute for its own signals.

This is consistent with how Google treats robots.txt (access control, not ranking) and sitemap.xml (crawl discovery, not ranking). Neither file improves your position in search results on its own; they affect whether and how Google can access your content. llms.txt is not in the same category as those files from Google's perspective — it simply is not part of its pipeline at all.

Verdict on Goal C: No effect on Google rankings. This is confirmed, not speculative.

Who has published llms.txt? (Goal D context)

The adoption signal is meaningful. As of April 2026, some of the most widely-used developer platforms have published llms.txt files:

• Anthropic (docs.anthropic.com) — one of the earliest adopters, unsurprisingly.
• Cloudflare — large-scale documentation site with multiple product sections.
• Stripe — published at stripe.com/docs/llms.txt, comprehensive API coverage.
• Vercel — Next.js and deployment documentation.
• Mintlify — popularized the llms-full.txt companion convention.
• Perplexity — notable given they are an AI search engine themselves.

The common thread is developer-facing documentation. These companies are not publishing llms.txt because they expect a Google ranking boost. They are publishing it because their users ask technical questions in AI assistants, and they want those assistants to have accurate context. That is the use case the file solves.

For Goal D (developer tooling), this adoption pattern is the strongest signal that llms.txt is working in the real world. When Cursor loads your llms.txt before helping a developer use your API, the developer gets better answers, fewer hallucinated endpoints, and faster onboarding. The companies above have judged that ROI to be positive.

Verdict on Goal D: Works concretely for developer tooling and documentation pipelines.

How to measure whether it is working

No single dashboard answers this today, but you can triangulate:

• Server logs. Filter for requests to /llms.txt and /llms-full.txt. Look for user-agents: GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Applebot-Extended. Frequency and recency of these hits tells you whether AI crawlers are actively interested.
• Referrer traffic. Watch for referrers from chat.openai.com, claude.ai, perplexity.ai, and similar domains. An increase after publishing llms.txt is not proof of causation, but it is worth tracking.
• AI citation monitoring. Tools like Profound and Otterly (as of early 2026) track how often and how accurately your brand is mentioned in AI-generated responses. These are early-stage but growing.
• Manual spot-checks. Ask Claude, ChatGPT, and Perplexity a question that your site should answer authoritatively. Note whether the answer is accurate, whether your pages are cited, and whether the quality improves after you publish or improve your llms.txt.
• Developer feedback. If your site serves developers, ask them directly: "When you ask an AI tool about [your product], do you get accurate answers?" This qualitative signal is often the most actionable.

Verdict

llms.txt works for what it was designed to do: giving AI systems a curated, structured summary of your site so they can ground their answers in accurate content. It works best for:

• Developer documentation sites
• API and SaaS products with technical buyers
• Any site whose users regularly ask AI assistants questions about the product

It does not work as a Google SEO lever. If that is your goal, you are in the wrong place — focus on structured data, Core Web Vitals, and editorial backlinks.

The cost-benefit calculation is asymmetric: a few hours to write a well-curated file, no downside, meaningful upside for LLM-adjacent discovery. For any developer-facing site, publishing llms.txt is a straightforward yes.

Next steps

• How to create an llms.txt file — templates and deployment guide for every common stack.
• Generator — fill a form and download a spec-compliant file in minutes.
• Validator — check your existing file against the spec.
• Best practices — what separates a useful llms.txt from noise.