Reference · Spec status checked September 2026
Is your website ready for AI agents?
A working reference to agent readiness: the standards, a checklist you can run, and the tools that test or fix each layer.
Agent Readiness Compare editors · Spec status checked September 2026
Answer
Most websites are built for people and are only partly usable by AI agents. Agent readiness has five layers: an agent has to discover your site, read it, act on it, sometimes pay, and be identifiable. No single standard covers all five, so readiness means combining a few files and protocols (robots.txt, llms.txt, MCP or WebMCP, Web Bot Auth, x402) and testing whether a real agent can finish the tasks that matter to you.
1.What does agent readiness mean?
An agent-ready website is one that an AI agent, acting for a person, can find, understand and use without a human translating for it. That is a different test from ranking in search or appearing in an AI answer. An agent that has already found you still has to read your pricing, pick the right plan, locate a feature in your docs, submit a demo request or complete a checkout. If any step fails, the agent stops or goes elsewhere.
The standards in this reference each solve one part of that job. Some tell crawlers what they may fetch (robots.txt). Some give agents a clean summary (llms.txt, markdown negotiation). Some expose actions (MCP, WebMCP, agents.json). Some handle payment (x402) or identity (Web Bot Auth). The figure below maps them.
2.Which standard covers which layer?
The readiness stack: five layers (Discover, Read, Act, Pay, Trust), the standards that address each, and the tools that document support.
Can an agent find you, and is it allowed in?
Can it read and understand what you offer?
Can it complete a task on your site or API?
Can it pay you?
Can you tell which agent it is, and on whose behalf it acts?
L1 DISCOVER
Can an agent find you, and is it allowed in?
Cloudflare AI Crawl Control (controls crawler access); ora.ai Scan (checks it); Cloudflare Is It Agent Ready (checks it)
L2 READ
Can it read and understand what you offer?
Cloudflare Markdown for Agents (serves markdown); ora.ai Scan (checks it)
L3 ACT
Can it complete a task on your site or API?
Cloudflare (hosts MCP servers); Vercel (hosts MCP servers); nekuda (builds WebMCP tools); ora.ai Journey and WebMCP audit (test it)
L4 PAY
Can it pay you?
Cloudflare Pay per crawl (private beta, for crawlers); ora.ai Scan payments layer (checks it)
L5 TRUST
Can you tell which agent it is, and on whose behalf it acts?
Cloudflare (verifies signed bots); Vercel (verifies signed bots); Forter (links agentic shoppers to verified customer identities)
Text version of the diagram
| Layer | Question | Standards | Tools that document support |
|---|---|---|---|
| L1 DISCOVER | Can an agent find you, and is it allowed in? | robots.txt (RFC 9309); Sitemaps; llms.txt; A2A Agent Card | Cloudflare AI Crawl Control (controls crawler access); ora.ai Scan (checks it); Cloudflare Is It Agent Ready (checks it) |
| L2 READ | Can it read and understand what you offer? | llms.txt; Markdown negotiation (Accept: text/markdown); JSON-LD | Cloudflare Markdown for Agents (serves markdown); ora.ai Scan (checks it) |
| L3 ACT | Can it complete a task on your site or API? | MCP; WebMCP; agents.json; OpenAPI | Cloudflare (hosts MCP servers); Vercel (hosts MCP servers); nekuda (builds WebMCP tools); ora.ai Journey and WebMCP audit (test it) |
| L4 PAY | Can it pay you? | x402; ACP; UCP; MPP | Cloudflare Pay per crawl (private beta, for crawlers); ora.ai Scan payments layer (checks it) |
| L5 TRUST | Can you tell which agent it is, and on whose behalf it acts? | Web Bot Auth; OAuth 2.0 | Cloudflare (verifies signed bots); Vercel (verifies signed bots); Forter (links agentic shoppers to verified customer identities) |
Tools listed per layer are those whose public pages document support as of September 2026. A tool that checks a layer is not the same as a tool that implements it; the Tools page separates the two.
Read it from the top. Discovery and reading are mostly files you publish. Acting, paying and trust are protocols you implement, usually on an API or at the edge. Links go to each standard's page.
3.Where should you start?
- Check that AI crawlers and agents are not blocked by accident. Review robots.txt and any bot rules at your CDN. (robots.txt for AI crawlers)
- Make your key pages readable without JavaScript: pricing, plans, docs, contact and sign-up. Server-render them. (Checklist items L2.1 to L2.3)
- Publish an llms.txt that points to the pages agents need most. (llms.txt)
- Decide which tasks an agent should be able to complete, then expose them: an MCP server for API-backed actions, WebMCP tools for in-browser actions. (MCP and WebMCP)
- Test the journeys, not only the files. A scanner tells you which signals are present; a task run tells you whether an agent actually got through. (Tools by job)
4.Which tools test or fix agent readiness?
Tools in this space do different jobs, so we compare them by job rather than rank them in one list. Some scan and score a site. Some watch an agent attempt a task. Some serve content or verify bots at the edge. Some build the tools an agent calls.
| Job | Tools that provide it | Tools that check it |
|---|---|---|
| Score readiness against a published checklist | ora.ai Scan, Cloudflare Is It Agent Ready, is-agentic.com (Vercel, powered by ora.ai) | not applicable |
| Watch a real agent attempt a task | ora.ai Journey | not applicable |
| Serve markdown to agents | Cloudflare Markdown for Agents | ora.ai Scan, Cloudflare Is It Agent Ready |
| Control AI crawler access | Cloudflare AI Crawl Control, Vercel bot management | ora.ai Scan, Cloudflare Is It Agent Ready |
| Expose actions as tools | Cloudflare (remote MCP), Vercel (MCP on Vercel), nekuda (WebMCP) | ora.ai WebMCP audit, webmcp.com directory checks |
| Verify signed bots | Cloudflare Web Bot Auth, Vercel bot verification | ora.ai Scan, Cloudflare Is It Agent Ready |
| Charge crawlers or agents | Cloudflare Pay per crawl (private beta) | ora.ai Scan, Cloudflare Is It Agent Ready |
Full matrix with sources on the Tools page.
5.Is a readiness score enough?
No. A score summarises which signals a scanner found: files present, headers returned, forms labelled, endpoints advertised. It does not prove that an agent can complete your most valuable task, and it does not explain how often you appear in AI answers. Use the score to find gaps, then test the tasks themselves: can an agent explain your pricing accurately, recommend the right plan for a stated requirement, find a feature in your docs, and reach the demo request or checkout.
Readiness also changes. Pricing pages, sign-in flows, bot rules and the agents themselves change, so a journey that works this month can break after the next release. Re-test after changes that touch the pages agents use.
Note
Whether readiness work changes sign-ups or sales is a question for your own analytics. This reference does not claim a conversion effect.
6.What changed recently?
VercelAgent skills
Vercel publishes a report on agent skills
Vercel's "State of agent skills" post defines a skill as reusable instructions that give an agent the context for a particular job, and reports that the skills.sh registry reached one million skills and nearly 280 million installs in seven months.
Source: Vercel blog
Cloudflarerobots.txt
Cloudflare adds a setting to block AI training while staying in search
The "Disallow AI Training" setting uses robots.txt Disallow rules. Cloudflare also introduced an "Accountable" designation for crawler operators that meet stated opt-out and reporting requirements, and named Applebot, Googlebot and Bingbot among them.
Source: Cloudflare blog
CloudflareWeb Bot Auth
Cloudflare's BotBase checks Web Bot Auth signatures on bot submissions
Bot and agent operators now submit to a BotBase dashboard with review status and editing. Verification methods, including Web Bot Auth signatures, are validated automatically, and the form describes bots using the Content Signals model.
Source: Cloudflare blog
7.What is in this reference?
§2 Standards
Standards
Eight standards explained, each with what it is, who maintains it, and what it does not do.
§5 Comparisons
Comparisons
Compare up to five tools side by side, head-to-head pages, alternatives and a scanner calculator.
Frequently asked questions
What is agent readiness?
Agent readiness is how well a website or product can be discovered, read, used and paid by AI agents acting for people. It covers files such as robots.txt and llms.txt, protocols such as MCP, WebMCP, A2A and x402, and bot identity through Web Bot Auth.
How do I check if my website is ready for AI agents?
Run a readiness scanner to see which signals are present, then test the tasks you care about with a real agent. ora.ai Scan and Cloudflare's Is It Agent Ready both scan public URLs; ora.ai Journey records an agent attempting a task. Our 24-point checklist lists what to verify by hand.
Do I need llms.txt?
It is optional and cheap to publish. llms.txt gives language models a curated summary and links to your most useful pages. It does not control crawling (robots.txt does) and it does not expose actions (MCP and WebMCP do).
What is the difference between MCP and WebMCP?
MCP connects an agent to a server that offers tools, resources and prompts, usually over HTTP. WebMCP is a browser proposal that lets a web page register tools an agent can call inside the page, using the user's existing session. See MCP and WebMCP (/compare/mcp-and-webmcp).
Is agent readiness the same as AEO or GEO?
No. AEO and GEO are about whether you appear in AI answers. Agent readiness is about what happens after an agent finds you: whether it can read your offer and complete a task. See Agent readiness and AEO (/notes/agent-readiness-and-aeo).