llms.txt explained: what it is, an example, and whether it works

llms.txt is a proposed Markdown file, served at /llms.txt, that summarizes a website for language models and AI agents and links to its most useful pages. Jeremy Howard of Answer.AI proposed it on September 3, 2024, and published version 2 of the spec in August 2026. It's a reading aid for machines. It controls nothing.

The honest bottom line: Google Search ignores it, and Google says so in its own guide. No major AI company documents that its crawlers read other sites' llms.txt files. And in an Ahrefs study of 137,210 domains, 97% of the files got no requests at all in May 2026. The file is cheap and harmless, so publish one if it helps AI tools that do read it, such as coding agents. Don't expect it to change whether ChatGPT or Google cite you.

I publish one for Inkie. Below: the exact format with a small example, Inkie's real file, how to make your own, and what to work on instead if you want to get cited.

What is llms.txt?

The spec calls it "a proposal to standardise on using an /llms.txt file to provide information to help agents use a website." HTML pages wrap their content in navigation, scripts and ads, and an agent with a limited context window wastes tokens stripping them out. A short Markdown file gives it a summary and a list of the pages worth reading.

The spec also suggests a Markdown copy of each important page at the same URL plus .md, and since v2, a rel="describedby" link that points a page to the llms.txt covering it.

Who it's for matters. The spec says llms.txt files "are used most heavily for software documentation, where coding agents follow them to find API references and tutorials." The idea that it lifts you in AI search came later, from the SEO industry, as Ahrefs notes. It also isn't a formal standard, just a proposal in a public GitHub repository.

The llms.txt format, per the spec

The v2 spec lists these parts, in this order, as Markdown:

1. An optional byte-order mark (BOM).
2. An H1 with the name of the project or site. This is the only required part.
3. A blockquote with a short summary "containing key information necessary for understanding the rest of the file."
4. Zero or more Markdown sections of any type except headings, such as paragraphs and lists.
5. Zero or more sections under H2 headings, each holding a "file list." Each list item needs a Markdown link, [name](url), and can add a colon and a note.

Three details trip people up:

• The file can sit at /llms.txt or at a subpath such as /docs/llms.txt. It covers the URLs under its path, and agents should use the most specific file.
• An H2 named Optional holds secondary links "an agent can skip when a shorter context is needed." In v1, tools that expanded the file into context left it out. The v2 changes page drops that meaning, so it's now only a convention.
• The spec doesn't define llms-full.txt. Docs platforms such as Mintlify generate one with the full text of the docs, and xAI publishes one for its developer docs. It's a separate, much larger file.

A complete llms.txt example

Here's a small, complete file for a made-up invoicing company, using every part of the spec.

# Example Invoicing

> Example Invoicing is invoicing software for freelancers and small agencies in the US and Canada. It sends invoices, takes card payments through Stripe and emails reminders to late payers. It does not run payroll.

Prices on the pricing page are current and in US dollars. Older blog posts may quote old prices.

## Product

- [Pricing](https://www.example.com/pricing.md): Plans, limits and what each includes
- [Features](https://www.example.com/features.md): Invoices, recurring billing, reminders and reports
- [Integrations](https://www.example.com/integrations.md): Stripe, QuickBooks Online and Zapier
- [Refunds](https://www.example.com/legal/refunds.md)

## Optional

- [Blog](https://www.example.com/blog): Guides on invoicing and getting paid

The blockquote says what the company sells, to whom, and one thing it doesn't do, so an agent that reads only that line can still answer "does it run payroll?" The .md links point to Markdown copies of the pages.

For more llms.txt examples, read the real files from OpenAI, Anthropic, Google's Gemini API docs and FastHTML, the spec's own example. All four are developer docs.

A real example: Inkie's llms.txt

Inkie serves its file at inkieai.com/llms.txt. Here's a shortened copy as served on October 24, 2026. Lines marked ... are cut.

# InkieAI

> InkieAI tracks how ChatGPT, Gemini, Perplexity, Google AI Overviews and Google AI Mode answer the questions customers ask, shows whether a brand is named, recommended or cited, and turns the gaps into page fixes and new content. It then checks that the work is live and measures later answers.

InkieAI is AI visibility tracking and GEO (generative engine optimization) software for B2B and B2C brands, built for founders and small teams. It is made in Stockholm, Sweden.

...

No one can guarantee that an AI service will mention or cite a brand, and InkieAI does not. It reports observed changes without claiming causation.

## Plans

- Start: $99 a month or $948 a year. 1 brand, 30 tracked questions, weekly monitoring, 10 content credits a month.
...

## Pages

- [Pricing](https://inkieai.com/pricing): Plans, trial terms and plan questions.
...

## Guides

- [AI crawlers explained: OAI-SearchBot, GPTBot, ClaudeBot and what to allow](https://inkieai.com/blog/ai-crawlers-robots-txt): OAI-SearchBot, GPTBot, ClaudeBot and PerplexityBot do different jobs. ...
...

## Optional

- [Blog](https://inkieai.com/blog): Every guide and article, newest first.
- [Sitemap](https://inkieai.com/sitemap.xml): Every indexable public URL.

Two choices in it are worth copying. The plan lines come from the same data as the pricing page, and the Guides section from the blog's own list, so the file changes when they do. And it states facts, including that nobody can guarantee an AI mention, without giving the model instructions.

The comment at the top of the code that builds it is blunt: "Google ignores the file and other crawlers rarely read it, so it is kept short and accurate rather than treated as a visibility lever." That's my position. Once generated, it costs nothing to keep, and I don't expect it to change any AI answer about Inkie.

How to create an llms.txt file

Whichever route you pick, the hard part is the writing: an accurate summary and a short list of the pages that answer real questions.

By hand

1. Write an H1 with your company or product name.
2. Write a blockquote of one to three sentences: what you sell, who it's for, and where. Add one thing you don't do if buyers often get it wrong.
3. Pick the pages that answer what buyers ask, such as pricing, features, integrations, docs and policies. The file should fit in an agent's context, so skip the rest.
4. Group the links under H2 headings, give each a short note, and put secondary pages under ## Optional.
5. Save it as llms.txt at your site root and confirm it loads as plain text.

With an llms.txt generator

Generators crawl your site and draft a file, and some platforms do it for you. According to the spec's integrations list, Mintlify and GitBook generate one for docs sites, and Wix does for every Wix site. Treat any generated file as a draft. A generator picks pages by what it finds, not by what buyers ask, so cut pages nobody needs and check every link.

On WordPress

Yoast SEO generates an llms.txt file, including in its free version. Yoast says the file "is generated and refreshed automatically," and you can let Yoast pick the pages or choose them yourself. Yoast's help article shows where to switch it on, and AIOSEO has a generator too. Yoast says the file helps AI tools "understand your site better" but cites no data. On other platforms, check your CMS or upload a static file to the site root.

Generated by code from your own pages

If your site is built in code, generate the file from the same data your pages use. Inkie does this with a Next.js route at /llms.txt that builds its plan and guide lists from the pricing and blog data, then returns plain text. When a price or a guide changes, the file changes on the next deploy.

Checklist before you publish

• It loads at /llms.txt with a 200 status as plain text, not an HTML page or a soft 404.
• It starts with exactly one H1, followed by a blockquote that says what you sell and to whom.
• Every link returns 200 at its canonical URL.
• Every fact matches the live page it summarizes, prices above all.
• It holds facts and links, with no instructions to AI models. Ahrefs saw a crawler named prompt-injection-survey fetching llms.txt files and recommends treating the file like code.
• Someone owns it, or a script regenerates it.

How to check your file

I didn't find an official llms.txt validator on llmstxt.org. Chrome's Lighthouse has an experimental llms.txt audit that flags server errors, but its docs don't say it checks the format. The spec suggests a better test: "Test your file by asking an agent questions about your content, giving it only your llms.txt as a starting point."

llms.txt vs robots.txt vs sitemap.xml

The names look alike, so people expect them to work alike. They don't.

The spec draws the line itself: "While sitemaps list all pages for search engines, llms.txt offers a curated overview for LLMs." So llms.txt can't block a crawler or let one in. If an AI crawler can't reach your pages, the fix is in robots.txt, your CDN or your firewall, and my guide to AI crawlers and robots.txt has three files to copy.

Does llms.txt work? What the evidence says

If "work" means AI tools get a clean summary of your site when they fetch it, yes. If it means more citations in ChatGPT, Perplexity or Google, nobody has shown that. Here's what I relied on, checked in October 2026.

What the Ahrefs data shows

Ahrefs looked at the 137,210 domains in its web analytics product with traffic in May 2026. 38,360 of them, 28%, served a valid llms.txt, and 97% of those files got no requests that month. The other 3%, about 1,100 domains, drew roughly 22,000 requests, and named AI tools made 19.5% of them.

Inside that 19.5%, AI agents made 10.5% of requests, training crawlers 5.3%, AI assistants such as ChatGPT-User 2.5%, and AI retrieval bots such as OAI-SearchBot and PerplexityBot 1.1%. GPTBot fetched the most of any AI bot, and Claude-Code, Anthropic's coding agent, out-fetched every AI search bot and assistant. Slackbot fetched more llms.txt files than PerplexityBot did. AI bots made zero requests for files that didn't exist.

Ahrefs notes its customers skew more technical and SEO-aware than the web at large, and that a fetch isn't proof anything used the file. It also sells the bot analytics product behind the study.

What the Otterly experiment shows

Otterly, which sells AI search monitoring, put an llms.txt at the root of one test site, linked it from the homepage, and logged AI-bot traffic for 90 days. The file got 84 visits while the site's average page got about 265. Otterly saw no noticeable rise in overall AI-bot activity and has since dropped its llms.txt check from its GEO audit. It's one site, so treat it as a data point.

Where Google takes both sides

Google Search ignores llms.txt, Chrome's Lighthouse audits for it, and Google publishes one for its Gemini API docs. That fits once you separate the jobs. Helping a coding agent read your API docs is one thing. Getting into AI answers is another. Only the first has evidence behind it, and that evidence is fetch counts.

I haven't found a published study that adds llms.txt to a set of sites and measures whether AI answers then name, recommend or cite them more often. The studies above count requests, which makes a citation effect unlikely but doesn't measure one.

Should you publish one?

Yes, if your CMS does it or it takes you an hour. No, if it pushes out real work. It's worth more if you have developer docs or an API, because coding agents are the readers with the clearest record of fetching these files.

The real risk is a wrong file. If it lists last year's prices, any agent that reads it gets last year's prices. Generate it from live data, or give it an owner.

What to do instead to get cited

If the goal is to get named, recommended or cited in AI answers, here's where I'd spend that hour. None of it guarantees a mention. Nothing does.

1. Let the search crawlers in. OpenAI says sites that opt out of OAI-SearchBot "will not be shown in ChatGPT search answers."
2. Stay eligible in Google. Google says "there are no additional requirements to appear in AI Overviews or AI Mode." A page must be indexed and eligible to show with a snippet.
3. Answer buyer questions on real pages: pricing, who it's for, how you compare, what you don't do.
4. Check that the outside sources AI answers cite for your category, such as review sites, describe you accurately.
5. Measure answers, not files. Ask the same buyer questions more than once and record who gets named and which pages get cited.

The full playbook is in how to get cited and recommended by ChatGPT, and GEO vs SEO vs AEO covers how much of this is ordinary SEO. To see where you stand first, check what ChatGPT and four other AI services say about you. Inkie's free report runs 5 buyer questions across ChatGPT, Gemini, Perplexity, AI Overviews and AI Mode, with no card needed.

Frequently asked questions

How do I create an llms.txt file for my website?

Write a Markdown file with an H1 naming your site, a short blockquote summary, and H2 sections that list your most useful pages as links with brief notes. Save it as llms.txt at your site root and check that it loads as plain text. On WordPress, Yoast SEO and AIOSEO can generate one, and Wix generates one automatically.

Does Google use llms.txt?

Not in Search. Google's AI optimization guide says llms.txt files "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them," and the guide covers AI Overviews and AI Mode. Chrome's Lighthouse has an experimental llms.txt audit, but that's a developer tool, not a ranking signal.

Does ChatGPT read llms.txt?

OpenAI doesn't document that its crawlers read other sites' llms.txt files. In Ahrefs' May 2026 data, GPTBot, OpenAI's training crawler, was the most frequent AI fetcher of these files, while OAI-SearchBot, which serves ChatGPT search, made few requests. A fetch doesn't show the file was used.

Is llms.txt an official standard?

No. It's a proposal by Jeremy Howard, published at llmstxt.org and open for input on GitHub, with version 2 released in August 2026. No major AI company has committed to reading other sites' files, though several publish their own.

What is llms-full.txt?

It's a single Markdown file with the full text of a site's documentation, usually served next to llms.txt. The llms.txt spec doesn't define it. Docs platforms such as Mintlify generate both files for the docs they host.

Your next step

If you don't have an llms.txt, write one in an hour or switch on your CMS's generator, then move on. First, though, open your robots.txt and CDN settings and confirm OAI-SearchBot, PerplexityBot and Googlebot can reach your pages. Blocking those has documented effects on AI search. llms.txt has none on record.

Then find out what AI answers say about you today. The free AI visibility report checks 5 buyer questions across ChatGPT, Gemini, Perplexity, Google AI Overviews and AI Mode, and shows whether each answer names you, recommends you or cites your pages.