Django 中的 llms.txt

Django 的四种做法:静态文件(通过 Nginx 提供)、返回 HttpResponse 的简单视图、从数据库模型生成的动态视图,或面向高流量站点的缓存视图。

最近更新:

选项 1:通过 STATICFILES_DIRS 提供静态文件

Django 的 staticfiles 框架会从以下位置复制文件: STATICFILES_DIRSSTATIC_ROOT 当您运行 collectstatic。然后您可以配置 Nginx 直接提供该文件,但请注意,该网址将位于您的 STATIC_URL 前缀(例如 /static/llms.txt),而不是放在根目录。

static/llms.txt, approach notes
# Django static file approach
#
# 1. Place llms.txt in one of your STATICFILES_DIRS:
#    myproject/
#    ├── static/
#    │   └── llms.txt   ← add this
#    └── myapp/
#
# 2. In settings.py, make sure STATICFILES_DIRS includes the static/ folder:
#    STATICFILES_DIRS = [BASE_DIR / "static"]
#
# 3. Run collectstatic to copy it to STATIC_ROOT:
#    python manage.py collectstatic
#
# NOTE: Django's staticfiles serve files under the STATIC_URL prefix (/static/ by default).
# This means the file will be at /static/llms.txt, NOT /llms.txt.
# Use the view approach below to serve at the root path /llms.txt.

要通过静态文件在以下路径提供内容: /llms.txt ,请使用 alias 指令,使其指向以下位置中收集的文件: STATIC_ROOT。请参阅下方部署部分中的 Nginx 代码片段。

方案 2:使用 HttpResponse 的自定义视图

最简洁的 Django 方案是创建一个最小视图,以正确的方式返回文件内容 content_type。此方法适用于任何部署目标,并会在以下精确路径提供文件: /llms.txt.

myapp/views.py
# myapp/views.py
from django.http import HttpResponse

LLMS_TXT_CONTENT = """# My Site

> One-sentence description of what my site or product does.

## Documentation

- [Getting Started](https://mysite.com/docs/start/): Install and configure in minutes.
- [API Reference](https://mysite.com/docs/api/): Full endpoint catalog with examples.

## Product

- [Overview](https://mysite.com/product/): Core features and capabilities.
- [Pricing](https://mysite.com/pricing/): Plans and billing details.

## Optional

- [Changelog](https://mysite.com/changelog/): Release history.
- [GitHub](https://github.com/my-org/my-repo): Source code.
"""


def llms_txt(request):
    return HttpResponse(
        LLMS_TXT_CONTENT,
        content_type="text/plain; charset=utf-8",
        headers={
            "Cache-Control": "public, max-age=3600, stale-while-revalidate=86400",
        },
    )

在根 urls.py:

myproject/urls.py
# myproject/urls.py
from django.urls import path
from myapp.views import llms_txt

urlpatterns = [
    # Serve at /llms.txt (root path)
    path("llms.txt", llms_txt, name="llms-txt"),
    # ... your other URL patterns
]

path("llms.txt", ...) 模式(无前导斜杠)匹配网址 /llms.txt。Django 会在匹配前移除开头的斜杠。

选项 3:从模型动态生成

对于文档页面存储在数据库中的网站,请根据 queryset 生成文件内容。使用 Django 内置的 cache_page 装饰器,以避免每次请求都访问数据库。

myapp/views.py, dynamic
# myapp/views.py
# Dynamic generation from database models
from django.http import HttpResponse
from django.views.decorators.cache import cache_page
from myapp.models import DocumentPage


# Cache for 1 hour, regenerated automatically when cache expires
@cache_page(60 * 60)
def llms_txt(request):
    docs = DocumentPage.objects.filter(
        published=True,
        include_in_llms=True,
    ).order_by("order").values("title", "slug", "summary")

    lines = ["# My Site", "", "> Documentation and API reference.", "", "## Documentation", ""]

    for doc in docs:
        url = f"https://mysite.com/docs/{doc['slug']}/"
        lines.append(f"- [{doc['title']}]({url}): {doc['summary']}")

    lines += [
        "",
        "## Optional",
        "",
        "- [Changelog](https://mysite.com/changelog/): Release history.",
    ]

    return HttpResponse(
        "\n".join(lines),
        content_type="text/plain; charset=utf-8",
    )

添加一个布尔字段 include_in_llms 添加到模型中,让编辑人员控制哪些页面出现在文件里。运行 验证器 ,以发现格式回归问题。

WSGI/ASGI 部署(Gunicorn、uWSGI、Nginx)

Django 视图适用于所有常见部署设置。该视图本身是同步且轻量的,无需异步处理。

nginx.conf, proxy to Gunicorn
# nginx.conf snippet, proxy to Gunicorn upstream
server {
    listen 80;
    server_name mysite.com;

    # Serve llms.txt directly from Gunicorn (Django view)
    location = /llms.txt {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_cache_valid 200 1h;
        add_header Cache-Control "public, max-age=3600";
    }

    # Optionally serve a pre-built static copy faster:
    # location = /llms.txt {
    #     alias /srv/mysite/static_root/llms.txt;
    #     add_header Content-Type "text/plain; charset=utf-8";
    # }
}
  • Gunicorn,开箱即用。该视图返回状态码为 200 且包含以下内容的响应: text/plain;无需特殊配置。
  • uWSGI, 与 Gunicorn 相同。如果你使用 uWSGI 的静态文件服务,可以将 /llms.txt 直接映射到以下位置的文件: STATIC_ROOTstatic-map = /llms.txt=/path/to/llms.txt.
  • ASGI(Daphne、Uvicorn),该视图无需修改即可兼容。Django 会自动在线程执行器中封装同步视图。
  • Nginx 缓存,添加 proxy_cache_valid 200 1h 添加到 Nginx location 块中,以便在代理层缓存响应,避免每次爬虫请求都访问 Django。

验证

部署后,确认文件已正确提供:

Verification
curl -I https://mysite.com/llms.txt
# Expected:
# HTTP/2 200
# content-type: text/plain; charset=utf-8

curl https://mysite.com/llms.txt | head -5
# Should print the first lines of your file

相关指南

来源