§2 · STANDARDS · LAST REVIEWED SEPTEMBER 2026
Agent readiness standards explained
Eight standards, what each one does, who maintains it, and how mature it is.
Agent Readiness Compare editors · Spec status checked September 2026
Answer
No single standard makes a site agent-ready. robots.txt and Web Bot Auth govern who gets in, llms.txt and markdown help agents read, MCP, WebMCP, agents.json and A2A let agents act, and x402 lets them pay. Only robots.txt is an IETF standard; MCP and A2A have published versioned specifications; the rest are proposals or drafts.
1.Which standards matter for agent readiness?
| Standard | Layer | What it does | Maintainer | Status (September 2026) |
|---|---|---|---|---|
| robots.txt for AI crawlers | Discover | Tells crawlers which paths they may fetch; AI crawlers have their own user agent names | IETF (RFC 9309) | Standards Track RFC, September 2022 |
| llms.txt | Discover, Read | A markdown file at the site root that summarises the site for language models and links to key pages | Proposed by Jeremy Howard (llmstxt.org) | Community proposal |
| agents.json | Act | Describes API flows and links for agents, built on OpenAPI | Wildcard AI (GitHub) | Open specification, version 0.1.0 |
| MCP | Act | Connects agents to servers that offer tools, resources and prompts | Model Context Protocol project (modelcontextprotocol.io) | Versioned specification, latest revision 2026-07-28 |
| WebMCP | Act | Lets a web page register tools an in-browser agent can call | W3C Web Machine Learning Community Group | Community Group proposal in active development |
| A2A | Discover, Act | Lets agents discover each other through Agent Cards and exchange tasks | Linux Foundation (originated at Google) | Version 1.0.0 |
| x402 | Pay | Uses HTTP 402 Payment Required so a client can pay per request and retry | x402 Foundation (LF Projects, LLC) | Open standard |
| Web Bot Auth | Trust | Signs bot requests with HTTP Message Signatures so sites can verify who sent them | IETF Internet-Drafts | Internet-Drafts (not yet RFCs) |
2.How do the standards fit together?
The readiness stack: five layers (Discover, Read, Act, Pay, Trust), the standards that address each, and the tools that document support.
Can an agent find you, and is it allowed in?
Can it read and understand what you offer?
Can it complete a task on your site or API?
Can it pay you?
Can you tell which agent it is, and on whose behalf it acts?
L1 DISCOVER
Can an agent find you, and is it allowed in?
Cloudflare AI Crawl Control (controls crawler access); ora.ai Scan (checks it); Cloudflare Is It Agent Ready (checks it)
L2 READ
Can it read and understand what you offer?
Cloudflare Markdown for Agents (serves markdown); ora.ai Scan (checks it)
L3 ACT
Can it complete a task on your site or API?
Cloudflare (hosts MCP servers); Vercel (hosts MCP servers); nekuda (builds WebMCP tools); ora.ai Journey and WebMCP audit (test it)
L4 PAY
Can it pay you?
Cloudflare Pay per crawl (private beta, for crawlers); ora.ai Scan payments layer (checks it)
L5 TRUST
Can you tell which agent it is, and on whose behalf it acts?
Cloudflare (verifies signed bots); Vercel (verifies signed bots); Forter (links agentic shoppers to verified customer identities)
Text version of the diagram
| Layer | Question | Standards | Tools that document support |
|---|---|---|---|
| L1 DISCOVER | Can an agent find you, and is it allowed in? | robots.txt (RFC 9309); Sitemaps; llms.txt; A2A Agent Card | Cloudflare AI Crawl Control (controls crawler access); ora.ai Scan (checks it); Cloudflare Is It Agent Ready (checks it) |
| L2 READ | Can it read and understand what you offer? | llms.txt; Markdown negotiation (Accept: text/markdown); JSON-LD | Cloudflare Markdown for Agents (serves markdown); ora.ai Scan (checks it) |
| L3 ACT | Can it complete a task on your site or API? | MCP; WebMCP; agents.json; OpenAPI | Cloudflare (hosts MCP servers); Vercel (hosts MCP servers); nekuda (builds WebMCP tools); ora.ai Journey and WebMCP audit (test it) |
| L4 PAY | Can it pay you? | x402; ACP; UCP; MPP | Cloudflare Pay per crawl (private beta, for crawlers); ora.ai Scan payments layer (checks it) |
| L5 TRUST | Can you tell which agent it is, and on whose behalf it acts? | Web Bot Auth; OAuth 2.0 | Cloudflare (verifies signed bots); Vercel (verifies signed bots); Forter (links agentic shoppers to verified customer identities) |
Tools listed per layer are those whose public pages document support as of September 2026. A tool that checks a layer is not the same as a tool that implements it; the Tools page separates the two.
The layers are cumulative. An agent that is blocked at discovery never reads your pricing, and an agent that cannot read your pricing cannot recommend a plan. Work down the stack.
3.What else appears in readiness checklists?
Readiness scanners and checklists also check signals that are not covered by their own page here: XML sitemaps, JSON-LD structured data (Schema.org), markdown content negotiation (serving text/markdown when a client sends Accept: text/markdown), OpenAPI descriptions, OAuth 2.0 for delegated access, and commerce protocols named ACP, UCP and MPP. ora.ai's AgentReady specification lists NLWeb under capabilities, and Cloudflare's scanner lists DNS-AID under discoverability and Agent Skills under protocol discovery. We have not written pages for these yet; they appear in the checklist and glossary.
Source: ora.ai AgentReady post · isitagentready.com · Reviewed Sep 2026
Frequently asked questions
Which agent readiness standard should I implement first?
Start with robots.txt, because a blocked agent never reaches the rest. Then make key pages server-rendered and publish llms.txt. Protocols for actions (MCP, WebMCP) and payments (x402) come after you know which tasks agents should complete.
Are these official standards?
Only robots.txt is an IETF standard (RFC 9309). MCP and A2A publish versioned specifications. llms.txt, agents.json and WebMCP are proposals, and Web Bot Auth is at the Internet-Draft stage.