Skip to content
SIA "OATS"
All writing Build notes

llms.txt: What It Is and Whether Your Site Needs One

16 November 2026 · 5 min read

In June 2026 Google wrote that Google Search doesn’t use llms.txt files, and that having one “will neither harm nor help” your rankings.1 That’s the plain answer for search. The file still has a use. It’s just smaller than the tools selling it suggest.

What the file is

llms.txt is a Markdown file at the root of a website, at /llms.txt. It gives an AI tool a short description of the site and a list of links to the pages worth reading. Jeremy Howard proposed it on the Answer.AI blog on 3 September 2024, and a second version of the proposal came out in August 2026.2

It’s still a proposal, so nobody is obliged to read it. The idea is that an agent opens the file first and follows only the links it needs, instead of digging through menus and scripts.

Who actually reads it

Google Search doesn’t read it. Its AI optimization guide lists llms.txt under things you can ignore and says that “Google Search itself doesn’t use them”.1 Chrome, which is also Google, is warmer. Its Lighthouse tool now has an llms.txt audit that calls the file “an emerging convention”. The audit only fails when the server errors on the request. A missing file is marked “Not Applicable”.3

OpenAI, Anthropic and Perplexity each publish a page about their crawlers and how robots.txt controls them. None of those pages says the crawler reads a site’s llms.txt.4 I couldn’t find any statement from these companies that ChatGPT, Claude or Perplexity uses the file when it answers questions about a business.

Where it does get used is software documentation. OpenAI, Anthropic and Google all publish an llms.txt for their own developer docs, and the proposal’s author says coding agents “use them reliably”.5 That’s his claim, not a measurement. It fits the design, though: a programmer’s assistant needs the right API page fast, and a curated list of links is exactly that.

What goes in one

Only one line is required, a heading with the site’s name. After that the proposal sets an order. First a quoted summary, then any plain notes, then sections of links under their own headings. Each link can carry a short note after a colon. A section called “Optional” holds links an agent can skip when it’s short on space.2

For a small business it might look like this:

# Kalna Joinery

> Made-to-measure kitchens and wardrobes from a workshop in Valmiera,
> Latvia. We install across Vidzeme.

## Services

- [Kitchens](https://example.lv/en/kitchens.md): oak and ash
- [Prices](https://example.lv/en/prices.md): ranges and what moves them

## Optional

- [Past projects](https://example.lv/en/projects.md): photos and notes

The company is made up. The .md links point to plain Markdown copies of the pages, which the proposal also suggests. Normal page links work too. Its own test is a good one: give an assistant only the file and ask it about your business.2

How it sits next to robots.txt and sitemap.xml

The three files do different jobs. robots.txt tells bots what they’re allowed to fetch. sitemap.xml lists every page you want a search engine to know about. llms.txt is a short, hand‑picked reading list for an agent that is helping someone right now.2

None replaces the others. A sitemap for a shop can run to hundreds of pages, often too much for an AI tool to read in one go. And the proposal pitches llms.txt as context for content robots.txt already allows. It grants no access of its own.

Does a small business site need one?

Need, no. Your Google visibility won’t change either way, and no major assistant has said it will treat you better for having one.1 4

It’s still cheap, and it can’t hurt. Wix generates and maintains the file automatically, and Yoast SEO can generate one on WordPress, so you may have one already.6 Open yoursite.com/llms.txt and read it. An automatic file can say less than two sentences you’d write yourself.

I’d spend the time in a different order. First make sure AI tools can load your pages and read the text without JavaScript, and that your company name, address and services are written out. Then ten minutes on an llms.txt is a fair trade.

The AI readability half of the OATS scan checks each of those. For llms.txt it asks for one thing: a file at /llms.txt that starts with a Markdown heading. If it’s missing or isn’t Markdown, the report lists “No llms.txt file” as a low‑impact finding, which is about the right weight for it.

FAQ

Does llms.txt help my Google rankings?

No. Google says Google Search ignores llms.txt files, so having one will neither harm nor help your visibility or rankings.

Do ChatGPT and Claude read llms.txt?

Neither company says so. Their crawler pages cover robots.txt and don’t mention llms.txt. Both publish one for their developer docs.

Sources and notes


  1. Google Search Central, “AI optimization guide”, mythbusting section, and the documentation update of 15 June 2026, “Clarifying guidance on llms.txt files”. ↩↩↩

  2. llmstxt.org, “The /llms.txt file”, v2, and its “Changes” page (v2, August 2026); Jeremy Howard, Answer.AI blog, 3 September 2024. Format, placement and the testing advice. ↩↩↩↩

  3. Chrome for Developers, Lighthouse agentic browsing audit “llms.txt”, last updated 5 May 2026. ↩

  4. OpenAI, “Overview of OpenAI Crawlers”; Anthropic’s support article on ClaudeBot, Claude‑User and Claude‑SearchBot; Perplexity docs, “Perplexity Crawlers”. Read 15 November 2026. ↩↩

  5. The llms.txt files at developers.openai.com, docs.anthropic.com and ai.google.dev, checked 15 November 2026. Quote from the v2 “Changes” page on llmstxt.org. ↩

  6. Wix Help Center, “Understanding Your Site’s llms.txt File”; Yoast, llms.txt feature page. ↩

Scan your site

Find out what attackers and AI assistants see on your page. Enter the address and get a report.

Write to me

or write directly: