Does llms.txt Actually Work? What Google Says

The short answer

No. Not in the way it's usually sold.

Google has said directly that you don't need new machine-readable files to show up in its generative AI features, and that crawling such a file doesn't mean the file gets treated specially. No major AI provider has published evidence that an llms.txt file improves how often you're cited.

That's the honest state of things as of mid-2026. If someone is charging you for an llms.txt implementation, you're paying for a text file.

We publish one at llmvisibilityindex.com/llms.txt anyway. It costs about twenty minutes to write and nothing to maintain. We just don't pretend it moves our numbers.

What llms.txt is

It's a proposed convention, not a standard. You put a markdown file at the root of your domain that summarises what your site is and links to the pages you consider important, in a form that's cheap for a language model to read.

Think of it as a table of contents written for machines. The idea borrows its shape from robots.txt, which is where a lot of the confusion starts.

A minimal one looks like this:

# Acme Analytics

> Acme Analytics measures product usage for B2B SaaS teams.

## Docs
- [Getting started](https://acme.com/docs/start): install and first event
- [API reference](https://acme.com/docs/api): endpoints and auth

## About
- [Pricing](https://acme.com/pricing): plans and limits

That's it. Headings, a summary, and annotated links.

Why it spread so fast

Two reasons, and neither is evidence that it works.

The first is that it feels right. Everyone in search has a decade of muscle memory around robots.txt and sitemap.xml, files you put at the root that genuinely change crawler behaviour. llms.txt looks like it belongs in that family, so people assumed it functions like one.

The second is that it's the rare AEO task with a clear finish line. Most of the work in getting cited by AI is slow and diffuse: earning accurate mentions elsewhere, keeping your facts consistent, restructuring pages so a model can lift an answer cleanly. Writing one file is a task you can close on a Tuesday. That makes it very easy to sell and very satisfying to ship.

What Google actually said

In May 2026 Google's Search Central team published its first official guidance on performing well in generative AI features. We went through the whole thing in our breakdown of Google's guide, but the relevant line is blunt:

"You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in generative AI search."

They add that they may crawl and index such files, but that "this doesn't mean that the file is treated in a special way."

Google went after two other popular tactics in the same section. Chunking your content into tiny fragments doesn't help, because their systems handle multiple topics on a page. Rewriting pages specifically for AI doesn't help either, since models understand synonyms and normal phrasing.

Worth being precise about the scope here: that guidance covers Google Search, meaning AI Overviews and AI Mode. It says nothing about ChatGPT, Claude, or Perplexity, and none of those have committed to reading llms.txt either. Anthropic, OpenAI and Perplexity have documented crawlers. None of them document this file.

So is it a scam?

That's too strong. It's a reasonable idea that got oversold.

There's a real problem underneath it. Models do work better with clean, well-structured, summarised content, and a lot of sites are genuinely hard to parse. llms.txt is one attempt at a fix. It just skipped the part where anyone with a model agreed to read it.

Compare it to schema markup, which does earn its keep. Schema is consumed by systems that have publicly committed to consuming it, and you can watch it change how your pages get represented. llms.txt has the format without the consumer.

The tell for a weak AEO vendor: their pitch centres on a file you upload or a score with no published formula. Ask what the score weights are and whether you can see the raw AI answers behind it. If both answers are vague, you're buying a dashboard.

Should you publish one?

Probably, and with low expectations. Here's how we'd think about it.

Reasons it's fine to do: it takes under an hour, there's no downside, it's a decent forcing function for writing a clear summary of your own site, and if the convention ever gets adopted you're already there.

Reasons not to bother: it does nothing measurable today, it's one more file to drift out of date, and every hour on it is an hour not spent on the things that do move citations.

If you do publish one, keep it honest and keep it current. A stale llms.txt listing pages you deleted is worse than none, because the one thing we know for certain is that it can be crawled.

Ours drifted, for what it's worth. It listed two blog posts out of twenty-six for weeks because it was hand-maintained and nobody regenerated it when new posts shipped. Generate it from your content, or accept that it'll rot.

What to do instead

The unglamorous list, in the order that pays off.

Check that AI crawlers can actually reach you. This is the one that silently wrecks everything downstream. If your CDN or WAF is throwing JavaScript challenges at GPTBot, PerplexityBot or ClaudeBot, no amount of file-writing helps. Plenty of sites blanket-blocked AI crawlers in 2023 and 2024 and forgot. Here's how to audit that with Cloudflare.

Make your answers liftable. A model quoting you wants a complete answer in one paragraph, not a claim assembled from five. That's a structural editing job, covered in how to write AI optimized content.

Fix how other sites describe you. Models synthesise across sources and hedge when those sources disagree. A surprising share of the work sits on review sites and roundups you don't control.

Measure it. Fixed non-branded prompt set, weekly, across several engines, tracking position and share of voice rather than just presence. The maths is in how an AI visibility score is calculated, and you can see it running live on the AI Visibility Index.

None of that closes on a Tuesday. That's rather the point.

Frequently asked questions

question: Does llms.txt work? answer: There is no published evidence that it improves how often AI systems cite you. Google has stated you do not need new machine-readable files to appear in its generative AI features, and that crawling such a file does not mean it is treated specially. No major AI provider documents reading it.

question: What is an llms.txt file? answer: A proposed convention, not a standard. It is a markdown file at your domain root that summarises what your site is and lists your important pages with short descriptions, formatted so a language model can read it cheaply. Its shape is borrowed from robots.txt, which is part of why people assume it behaves like one.

question: Should I still add llms.txt to my website? answer: It is reasonable to publish one if you keep expectations low. It takes under an hour, carries no downside, and positions you if the convention is ever adopted. Just do not treat it as an AEO lever, and generate it from your content so it does not go stale.

question: Is llms.txt the same as robots.txt? answer: No. robots.txt is a long-established standard that crawlers genuinely obey, controlling what they may fetch. llms.txt only borrows the idea of a root-level text file. No crawler is committed to reading it or changing behaviour because of it.

question: What should I do instead of llms.txt? answer: Confirm AI crawlers can fetch your pages without hitting bot challenges, restructure key pages so each answer is complete in a single paragraph, correct how third-party sources describe your brand, and measure citations weekly with a fixed non-branded prompt set.

Written by the PingAura Data Team, the team behind the LLM Visibility Index. The index tracks how brands rank in AI answers across 25 major industries in India.