llms.txt is a plain-text file, proposed by Jeremy Howard of Answer.AI in September 2024, that sits at your site root and points a language model at clean, machine-readable versions of your important pages (llmstxt.org). The standard pitch for adding one is that AI assistants will look for it, read it, and retrieve your content through it. That pitch is mostly wrong in 2026, and it is worth walking through why, because the honest version of the argument still ends with "publish one anyway."
Begin with adoption, since that is the number people quote first. A SE Ranking scan of 300,000 domains found llms.txt on roughly 10% of sites (SE Ranking), so the file is no longer exotic. However, publishers adopting a file and crawlers consuming it are two different measurements, and the consumption side is where the case breaks down. None of the major AI providers have confirmed that they use llms.txt. Google's John Mueller pointed out that you can read your own server logs and watch the AI services never request the file, and Google's Gary Illyes confirmed in mid-2025 that Google does not support it and has no plans to (Search Engine Journal). One study of correctly implemented files found that of 62,100 AI bot requests, 84 went to the llms.txt, about 0.1% (ppc.land). Meaning if you shipped one today expecting ChatGPT or Perplexity to fetch it, the logs would show they almost never do.
This is the point where the conventional take stops, usually with a verdict that llms.txt is dead. That verdict mistakes the surface for the mechanism. One layer down there is a real consumer already reading the file every day: developer tooling. AI coding assistants such as Cursor, GitHub Copilot, and Claude pull documentation into context at request time, and a well-formed llms.txt lets them fetch the right pages with far less wasted token budget (ppc.land). So the file is not speculative for everyone. If your site is documentation, an API, or a developer product, it is doing work right now. If your site is a marketing page, it is doing much less, and you should price it accordingly.
The reason the file is worth publishing in either case comes down to what it costs versus what it replaces. An llms.txt is a title, a short description, and a list of links to Markdown or plain-text versions of your key pages. A tool that supports it makes one fetch and gets a curated map of your site. When the file is absent, that same tool falls back to fetching rendered HTML and stripping navigation, ads, and boilerplate to infer the substance, and the fallback is lossy in precisely the cases that matter: client-rendered pages, decorative headings, links that lead to more unparseable pages. The file does not grant a new capability so much as it removes the guessing, which is why it is cheap to produce and hard to regret.
Within the MachineRead rubric, llms.txt is worth 5 of the 56 checked points and counts as one of the 8 default agent-readiness probes. I weight it modestly on purpose. Scoring a file heavily when most crawlers do not yet read it would overstate its present value, and the entire premise of the audit is to separate the gaps that block retrieval today from the ones that are cheap insurance against where retrieval is heading. llms.txt is squarely in the second category for most sites and the first category for developer tools.
In essence, my recommendation splits by what kind of site you run. If you run docs, an API, or a developer product, write the llms.txt now, because the coding assistants already consume it and the payoff is immediate. If you run a general site, still write it, but treat it as a low-cost hedge rather than a traffic source, since it takes under an hour, costs nothing to host, and closes one probe. What you should not do is let anyone sell you llms.txt as your entry into ChatGPT's answers, because the server logs do not support that claim yet. If the major crawlers do start reading it, and that remains plausible, you will already have it in place.
See also
- Methodology reference - documents the llms.txt probe and all 13 check groups
- Launch post - covers how llms.txt fits the four agent-consumable surfaces
- Agent integration guide - details the surface family
- Related: schema.org JSON-LD and ai-catalog.json