llms.txt redirects: what 113 files reveal
A redirect can preserve access during a migration, but it also adds latency and can move the effective trust boundary. Our directory records both requested and final URLs.
Last updated:
Thirty-three of 113 verified root URLs ended at a different URL after redirects. Thirty-one crossed to another origin. Redirects are not forbidden by the proposal, but recording the final destination is necessary for reliability, scope analysis, and incident review.
Key takeaways
- 29.2% of verified requested URLs redirected in this snapshot.
- 31 of the 33 redirects changed origin.
- Direct final URLs reduce latency, but migrations need careful compatibility.
How common were redirects?
The directory stores both the requested https://host/llms.txt and the final response URL. In the 3 August snapshot, 33 of 113 differed. Thirty-one final URLs had a different origin from the original request.
This count describes the file fetch path, not every link inside the file. It includes legitimate documentation migrations and host consolidation. Inspect individual entries in the verified directory before assigning a cause.
What does a cross-origin redirect mean?
It means the effective file is served by another scheme-host-port combination. A documentation brand may move to a platform domain, or a legacy hostname may redirect to its replacement. The client ultimately reads the final origin’s response and security policy.
That does not make the file invalid. It does make ownership and scope less obvious. If the final map links back to the intended documentation and the redirect is deliberate, it can be a sensible compatibility layer.
A maintenance policy for redirects
Store requested URL, final URL, status, and check date. Alert on new cross-origin moves because they deserve review. Keep long-lived inbound compatibility where external clients may know the old path, but update controlled references to the final canonical file.
Test for loops, excessive chains, downgrade to HTTP, unexpected HTML, and changes of content type. Run these checks outside the per-push production gate so a temporary third-party incident does not block unrelated releases.
Use the validator after following the redirect and the v2 migration checklist when changing your own file location.
Check how to find path-level files and compare redirect patterns in the verified directory.
What the benchmark cannot decide
The dataset records where the request ended, not why the redirect exists or whether an agent trusts it. It cannot label a migration good or bad automatically. It also provides no evidence of Search impact.
Are redirects allowed?
The proposal does not declare a redirected file invalid. Direct stable resources are simpler for clients.
Should I remove an old root redirect?
Only after confirming that no relevant client relies on it. Stable compatibility can outweigh one extra hop.
Why review cross-origin moves?
They change the server and policy delivering the effective file, so they deserve explicit ownership review.