What is llms.txt?

1.What is llms.txt?

llms.txt is a proposal, first published by Jeremy Howard in 2024, for a markdown file that helps language models use a website at inference time. Instead of making a model parse full HTML pages with navigation, scripts and layout, the site publishes a short, curated file that says what the site is and where the important content lives.

2.What goes in the file?

The proposal defines a fixed order:

1. An H1 with the name of the project or site. This is the only required section.
2. A blockquote with a short summary.
3. Optional paragraphs or lists with more detail.
4. Zero or more H2 sections, each containing a markdown list of links in the form [name](url), optionally followed by a colon and notes.
5. An H2 section named 'Optional' for links an agent can skip when context is short.

# Example Analytics

> Example Analytics is a product analytics tool for SaaS teams. Plans start with a free tier; paid plans are billed per tracked event.

## Product
- [Pricing](https://example.com/pricing.md): plans, limits and billing
- [Features](https://example.com/features.md): what each plan includes

## Docs
- [Quickstart](https://example.com/docs/quickstart.md): install and first event
- [API reference](https://example.com/docs/api.md): endpoints and authentication

## Optional
- [Changelog](https://example.com/changelog.md)

3.Where does the file go?

At the site root (/llms.txt) to cover the whole site. The current proposal also allows a file at a sub-path, for example /docs/llms.txt, covering only the pages under that path. The proposal also asks sites to offer clean markdown versions of pages at the same URL with .md appended (page.html.md) or with the extension replaced by .md.

4.What does llms.txt not do?

• It does not control access. Crawl permissions live in robots.txt.
• It does not expose actions. For tasks, see MCP and WebMCP.
• It is not guaranteed to be read. Whether a given agent or model fetches llms.txt is up to that agent.

5.How do you check it?

1. Fetch https://yourdomain/llms.txt and confirm HTTP 200 and a text or markdown content type.
2. Confirm the first line is an H1 and that the summary is accurate and current, including pricing wording.
3. Open every link. Linked pages should return 200 and, ideally, markdown.
4. Compare it with a readiness scanner result; ora.ai Scan and Cloudflare Is It Agent Ready both look for discovery files.

6.Where does it sit in the readiness stack?

Layers 1 and 2: it helps an agent find the right pages and read them cheaply.

Frequently asked questions