Does llms.txt actually work? Mostly, no. Server-log analysis across 137,000 domains found 97% of llms.txt files received zero requests in a single month. A separate study watching 83 sites for twelve weeks recorded OpenAI fetching llms.txt seven times, against 3,990 fetches of robots.txt on the same sites. One company is reading it at scale, and it is not the one you would guess. The file is still worth adding, for a reason that has nothing to do with citations.

What llms.txt is meant to do

Jeremy Howard proposed llms.txt in September 2024. The idea is simple and reasonable: a Markdown file at the root of your domain that tells an AI system what your site contains and where the important pages are, in a format a model can read without wading through navigation and scripts.

It sits alongside two companion formats. llms-full.txt contains your actual content rather than a map of it. ai.txt is a separate proposal covering training permissions.

Nothing about the idea is foolish. The question is whether anything reads it.

Adoption is growing fast from a tiny base

Originality.ai has tracked more than three million websites for a year. Between June 2025 and May 2026, llms.txt files went from 4,088 to 36,120 instances, an 8.8-fold increase. llms-full.txt grew 107-fold and ai.txt 99-fold, though both from almost nothing.

Among larger sites, Rankability scanned the Tranco top 1,000 in September 2026 and confirmed 9.3% serving an llms.txt or llms-full.txt file. Their earlier observations put it at 0.3% in June 2025 and 8.7% in June 2026, though they caution these are separate snapshots rather than a continuous series, because the method changed.

One honest wrinkle in that figure: 474 of the 1,000 domains returned inconclusive results, and they stayed in the denominator. So 9.3% is a floor rather than an estimate.

Adoption skews by sector. Travel sits at 16%, financial services at 15.4%, music streaming at 12.8%, ecommerce at 7.4%. Cloudflare, GitHub, WordPress.org, Adobe and Dropbox all publish one.

Who is actually reading it?

This is where the case falls apart.

Ahrefs analysed server logs from 137,000 domains and found that 97% of llms.txt files received zero requests in May 2026. Of the requests that did arrive, AI retrieval bots accounted for 1.1%. The largest requester category was SEO audit tools at 21.7%, followed by unidentified bots at 14.9% and general web crawlers at 13.1%.

In other words, the thing most likely to read your llms.txt file is a tool checking whether you have an llms.txt file.

A smaller study puts numbers on individual crawlers. EZY.ai watched 83 sites over twelve weeks from late April to mid July 2026 and counted fetches of robots.txt against fetches of llms.txt.

Read that Meta row twice. Meta-ExternalAgent fetched llms.txt 193 times and robots.txt only 172 times. It is the one major AI crawler genuinely reading the file, and it reads it more often than it reads the file everyone considers standard.

Perplexity never touched it once.

Two caveats on that table, because they matter. The counts are tiny, so single-digit differences are fragile. And EZY.ai sells llms.txt generation, which is a conflict worth naming even though their conclusion partly argues against their own product.

What have the platforms said?

Short version: almost nothing, and what exists is discouraging.

Google’s John Mueller said in June 2025 that no AI system currently uses llms.txt. Google published an AI search guide in May 2026 and added a note the following month confirming llms.txt is not required for Google Search. Chrome’s Lighthouse added an optional agentic-browsing audit for it, which is the closest thing to institutional endorsement anyone has managed.

OpenAI, Anthropic and Perplexity have made no public statement that they use it.

Rankability, who produced the adoption data, state the position plainly: publishing one does not establish ranking improvements, AI citations or training permissions.

The file that actually matters

Here is the part worth taking away, and it is not about llms.txt at all.

Every study above found the same thing incidentally: robots.txt is fetched constantly by every major AI crawler. On those 83 sites, Googlebot hit robots.txt 5,125 times, OpenAI 3,990 times, Anthropic 3,120 times. That is where AI access is decided today.

And getting it wrong has a measured cost. Research from Rutgers and Wharton published in April 2026 found that news publishers blocking AI crawlers through robots.txt lost roughly 7% of their weekly website traffic within six weeks.

So the asymmetry is stark. A misconfigured robots.txt measurably costs you traffic. A missing llms.txt costs you almost nothing, because almost nothing reads it.

If you are going to spend an hour on a text file this week, spend it confirming your robots.txt allows GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot, Google-Extended and Bingbot, and that your firewall is not quietly blocking them anyway.

So should you add llms.txt?

Yes, and it should take you ten minutes.

The case for it is not that it works. The case is that it costs nothing, Meta demonstrably reads it, the format is on a standards track, and if OpenAI or Anthropic adopt it next year you will already be compliant. That is reasonable insurance at a reasonable price.

What you should not do is pay anyone a meaningful sum to produce one, treat it as a GEO strategy, or believe a vendor who tells you it will get you cited. On current evidence that claim is ahead of the data.

A minimal file listing your main sections with one-line descriptions is sufficient. Generate it, upload it to your root directory, and move on to the work that does move the needle.

Where to actually spend the effort

If llms.txt is a ten-minute job, what deserves the afternoon?

Making sure AI crawlers can reach and render your pages. Writing openings that answer the question in the first sixty words. Putting specific numbers and named entities in your claims. Earning mentions on sites you do not own. Building review volume.

Those are the variables that keep appearing in every analysis of what cited pages have in common. None of them live in a text file.

Want to know which of your signals is weak?

We will run twenty buyer prompts across ChatGPT, Gemini, Perplexity and Copilot, check whether you appear, and tell you what is actually holding you back. It is rarely your llms.txt.

No charge, and the report is yours either way.

Frequently Asked Questions

Does llms.txt work?

Mostly no, on current evidence. Ahrefs analysed server logs from 137,000 domains and found 97% of llms.txt files received zero requests in May 2026. AI retrieval bots accounted for just 1.1% of the requests that did arrive. The largest single requester category was SEO audit tools.

Do any AI companies read llms.txt?

Meta does, at scale. Across 83 sites over twelve weeks, Meta-ExternalAgent fetched llms.txt 193 times — more often than it fetched robots.txt. Over the same period OpenAI fetched it 7 times, Anthropic 9 times, Googlebot 67 times, and Perplexity not at all.

Has Google confirmed it uses llms.txt?

No. John Mueller said in June 2025 that no AI system uses llms.txt, and Google added a note to its AI search guide in June 2026 confirming the file is not required for Google Search. Chrome’s Lighthouse includes an optional agentic-browsing audit for it, which is the nearest thing to official recognition.

How many websites have an llms.txt file?

Around 36,120 instances as of May 2026, up 8.8x from 4,088 a year earlier across a sample of three million-plus sites. Among the Tranco top 1,000 domains, 9.3% were confirmed serving one in September 2026 — a floor rather than an estimate, since nearly half the sample returned inconclusive results.

Is llms.txt or robots.txt more important?

robots.txt, by a wide margin. Every major AI crawler fetches it constantly, and it is where AI access is actually decided. Research from Rutgers and Wharton found publishers blocking AI crawlers via robots.txt lost roughly 7% of weekly traffic within six weeks. Misconfiguring robots.txt has measurable cost; omitting llms.txt does not.

Should I still add llms.txt to my site?

Yes, but treat it as ten minutes of cheap insurance rather than a strategy. It costs nothing, Meta reads it, the format is on a standards track, and you will already be compliant if other platforms adopt it. Do not pay a meaningful sum for one, and do not believe anyone who says it will get you cited.