Home / Blog # llms.txt blog In-depth articles on the llms.txt spec, AI crawlers, generative engine optimization, and practical implementation guides for developers and site owners. - 2026-08-10 ## [Adoption is not usage: reading the llms.txt studies properly](/blog/llms-txt-adoption-is-not-usage/) Why llms.txt adoption measures range from 10.13% to 51.8%: each study samples a different population, and publication is not consumption. [ Read article → ](/blog/llms-txt-adoption-is-not-usage/) - 2026-08-07 ## [Threat-modeling llms.txt prompt injection](/blog/llms-txt-prompt-injection/) Ahrefs observed a research user agent named prompt-injection-survey. The label does not prove an attack, but external agent context deserves supply-chain controls. [ Read article → ](/blog/llms-txt-prompt-injection/) - 2026-08-04 ## [Who fetches llms.txt? Agents, not AI search crawlers](/blog/who-actually-reads-llms-txt/) In Ahrefs logs for 137,210 domains, AI retrieval bots made 1.1% of llms.txt requests and agentic tools 10.5%. Fetching still does not prove use. [ Read article → ](/blog/who-actually-reads-llms-txt/) - 2026-08-01 ## [What Chrome's Lighthouse llms.txt audit actually checks](/blog/lighthouse-llms-txt-audit/) Lighthouse treats a 4xx response as Not Applicable. For a successful response it also checks minimum length, one H1 and one Markdown link, but not full v2 conformance. [ Read article → ](/blog/lighthouse-llms-txt-audit/) - 2026-07-29 ## [AI bots did not probe missing llms.txt files in one study](/blog/agents-do-not-look-for-llms-txt/) In Ahrefs' May 2026 sample, no AI bot requested a missing /llms.txt. The result is not universal proof, but it shows why publishing alone is not discovery. [ Read article → ](/blog/agents-do-not-look-for-llms-txt/) - 2026-06-10 ## [State of llms.txt adoption, June 2026: measured data](/blog/state-of-llms-txt-2026/) We fetch and parse /llms.txt on a fixed panel of 219 notable hosts every week. Around half serve the file, with sharp differences by sector. Raw data published. [ Read article → ](/blog/state-of-llms-txt-2026/) - 2026-04-22 ## [llms.txt vs robots.txt, what's the difference?](/blog/llms-txt-vs-robots-txt/) robots.txt publishes crawl rules; llms.txt offers compatible agents a curated map. Learn how the files differ in format, purpose and enforcement. [ Read article → ](/blog/llms-txt-vs-robots-txt/) - 2026-04-22 ## [GEO: Generative Engine Optimization, what it is and why it matters](/blog/what-is-geo/) Generative Engine Optimization (GEO) is the practice of optimizing content to appear in AI-generated answers. Learn how it differs from traditional SEO and where llms.txt fits in. [ Read article → ](/blog/what-is-geo/) - 2026-04-22 ## [llms.txt best practices, how to write a file AI systems actually use](/blog/llms-txt-best-practices-2025/) A practical DO/DON'T guide: use absolute URLs, write factual descriptions, keep it concise, and update regularly. Everything you need to write a file that works. [ Read article → ](/blog/llms-txt-best-practices-2025/) - 2026-04-22 ## [AI crawlers explained, how Perplexity, ChatGPT, and Claude read your site](/blog/ai-crawlers-explained/) AI crawlers differ from Googlebot: they build context, not just indexes. Understand what GPTBot, ClaudeBot, and PerplexityBot do and how llms.txt helps. [ Read article → ](/blog/ai-crawlers-explained/) - 2026-04-22 ## [llms.txt for API-first products](/blog/llms-txt-for-api-products/) Developer tools and API products benefit especially from llms.txt because developers ask AI assistants about APIs. A guide with a full example template. [ Read article → ](/blog/llms-txt-for-api-products/) - 2026-04-22 ## [llms-full.txt, what it is, when to use it, and how to create one](/blog/llms-full-txt-guide/) llms-full.txt is the companion file that inlines your full page content. Learn when it makes sense, how it differs from llms.txt, and who uses it. [ Read article → ](/blog/llms-full-txt-guide/) - 2026-04-22 ## [How to measure the impact of your llms.txt file](/blog/measure-llms-txt-impact/) No direct analytics dashboard exists yet, but you can triangulate. Track AI referral traffic, monitor bot fetch logs, and test citation accuracy manually. [ Read article → ](/blog/measure-llms-txt-impact/) - 2026-04-22 ## [llms.txt for documentation sites](/blog/llms-txt-documentation-sites/) Docs sites are the ideal use case for llms.txt. Learn what to include, what to exclude, and how to structure your file so AI assistants give accurate answers. [ Read article → ](/blog/llms-txt-documentation-sites/)