# AgentReady: Make your site readable by AI agents

> AgentReady is the open standard for agent readiness: a spec of the protocols, standards, and conventions a product should implement to be found, read, and used by AI agents, from discovery to completion. It does not promote any specific protocol or vendor, and it does not prescribe how scanners should weight or score compliance. Every requirement carries a stable identifier (AR-FIND-01 and so on) and a BCP 14 strength (MUST / SHOULD / MAY) assigned from real agent runs and a controlled experiment. Developed in collaboration with Ora, Vercel, Stripe, Auth0, Mintlify, Tavily, Google, Elementor, and Netlify.

Version: v2.0
Last updated: October 2026
License: MIT

## How we know

Numbers in this spec are tagged by source. [STUDIES] refers to research from the ora.ai research lab: more than 250,000 real agent runs and a controlled experiment that toggled one feature at a time, with the data behind each claim public at https://github.com/agentready-org/standard/tree/main/data; plus the Ora research reports cited next to their numbers. [ORA SCANNER] refers to agent-reachability checks across every scored site. Statements without a tag are recommendations rather than measured findings.

In the studies, two changes caused agents to fail: hiding answers behind JavaScript and blocking access. The spec follows the order in which agents encounter a site: find, read, then act. MUST marks what an agent fails without: those two changes, and, when agents should complete a task, an MCP server or a CLI to act through. Practices that measurably shaped where agents went or make acting reliable are SHOULD, emerging conventions are MAY. All recommendations share one prerequisite: the agent must be able to reach the page and find the answer in the returned HTML.

## Essentials

The short version. Find and read are the essentials every site needs. Act applies when agents should complete a task, not only read.

### find

- Publish first-party docs: The page agents ground answers on and name as the source.
- Allow agents in robots.txt and bot protection: The gate to search indexes and live fetches. About 1 in 10 scanned sites block AI crawlers entirely, and a bot manager can block agents the file allows.
- List pages in a sitemap.xml: Standard for crawlers. Agents fetch it in 4% of runs.

### read

- Put the answer in the initial HTML: Homepage and docs, server-rendered. A JavaScript-hidden answer was one of two changes that broke an agent.
- Return real HTTP status codes: 404 for a missing path, never 200. 429 with Retry-After when throttling.
- Link llms.txt, JSON-LD, and OpenAPI: 86–100% of those fetches arrived through a link, not a guessed path.
- Fence and language-tag code: When an agent’s answer included code, 61% of the time it lifted lines verbatim.

### act (if agents should complete a task, not only read)

- Auth an agent can finish: Self-serve keys or OAuth with discoverable metadata. The first gate of acting.
- A public API with OpenAPI: Typed operations, errors, and rate limits. The acting surface itself.
- An MCP server or a CLI: The interface an agent acts through. Ship at least one; without it, an agent cannot finish the task.
- WebMCP on pages agents should act through: In-page tools on document.modelContext, in the user's session. Complementary to a server MCP. Do not invent tools to chase a score.

## Check your site

Three public rankers score the site itself, each on a different surface. One report shows how you rank when agents search your market, and one harness runs real agents through tasks on your product. None of them is the spec. The rankers have a free browser score, the report runs on a free Tavily API key, and the harness is an open-source CLI that uses your own agent API key. is-agentic.com is the Vercel × Ora essentials reading for sites whose job is content agents can find and use: scan at https://is-agentic.com or run "npx is-agentic example.com". ora.ai is the full ranker for products agents transact with (API, MCP, payments, leaderboard): scan at https://ora.ai, run "npx ax audit https://example.com", or POST {"url": "example.com"} to https://ora.ai/api/scan. mintlify.com/score is Mintlify's Agent Score for documentation sites (29 checks on the Agent-Friendly Documentation Spec): scan at https://www.mintlify.com/score or run "npx mint score docs.example.com". avr.tavily.com is Tavily's report on who agents find when they search your market: run it at https://avr.tavily.com with a free Tavily API key. axis.run is Netlify's AXIS (Agent Experience Index Score), an open-source harness that runs 20+ coding agents through your API, MCP server, CLI, or skills, scores each run on four dimensions (goal achievement, environment, service, agent), and gates CI on a baseline: run "npx @netlify/axis init" then "npx @netlify/axis run". Docs: https://is-agentic.com/docs, https://ora.ai/docs, https://www.mintlify.com/blog/agent-score, and https://axis.run/scoring. Scanner checks and weights may change independently of this spec.

## Be reachable and citable

Agents can cite only pages they can discover and fetch. Pages blocked from crawlers do not appear in search indexes at all.

[STUDIES] When an agent cannot read your site, it answers anyway, from web search and other people's pages. Across 1,056 business sites, those answers were 3.7× as likely to contain none of the facts the user asked for, and agent-ready businesses were clearly recommended 1.9× as often (AX is the New AEO: https://ora.ai/blog/ax-is-the-new-aeo).

- AR-FIND-02 (SHOULD) Publish the answer as first-party docs: Makes you citable: the page an agent grounds its answer on and names as its source. [STUDIES] 82% of answers traced back to a page the agent fetched, and that page was a docs page 47% of the time, more than any other page type.
- AR-FIND-01 (MUST) Allow agents in robots.txt and bot protection: Declares which crawlers may fetch which paths, and lets verified agents through the bot manager. This is the gate to the search indexes. [ORA SCANNER] In the April 2026 report (~50,000 scanned sites), 88% shipped a robots.txt and 80% stayed reachable to every major AI crawler. About 1 in 10 blocked them entirely.
- AR-FIND-03 (SHOULD) List your pages in a sitemap.xml: Maps every page you publish for indexing. [STUDIES] Fetched directly in 4% of runs, one of the files real agents touch least. It is still standard for crawlers and search engines.
- AR-FIND-04 (MAY) Publish an AI Catalog and link it: One manifest at /.well-known/ai-catalog.json listing every agent-facing resource you ship, from MCP server card to llms.txt. Not measured in the studies. Link it from the homepage head, since 86–100% of discovery-file visits arrive through links. ARD registries such as Ora Directory index published catalogs so agents can search across sites.

An AI Catalog at /.well-known/ai-catalog.json lists every agent-facing resource (MCP server card, OpenAPI, GraphQL, A2A agent card, docs, llms.txt, skills); link it from the homepage head with <link rel="ai-catalog"> and an HTTP Link header. Agentic Resource Discovery (ARD) registries such as Ora Directory (https://ora.ai/directory) index published catalogs and answer search across them.

Publish answers on one canonical, current first-party docs page. Write the crawl policy as one rule for all agents (User-agent: *) instead of an allowlist of named crawlers. Model providers run separate crawlers per job: training crawlers (GPTBot, ClaudeBot), search-index crawlers (OAI-SearchBot, Claude-SearchBot), and user-initiated fetchers (ChatGPT-User, Claude-User). Each obeys its own rule, so you can block training without dropping out of search indexes or agent answers. A more specific group such as "User-agent: GPTBot" overrides the * group, and user-initiated fetchers may not consult robots.txt at all. Agents also find pages through agent search APIs such as the Brave Search API, Exa, Parallel, and Tavily, which run their own crawlers and indexes; the wildcard rule covers them, an allowlist of named crawlers does not. Bot protection is a second gate that robots.txt does not control: a WAF or bot manager that answers agents with a 403 or a challenge page blocks them however permissive the file is. Let verified agents through by the providers' published IP ranges or Web Bot Auth signatures, and throttle with a 429 instead of a challenge. [STUDIES] Sites that were not agent-ready blocked agents 2.1× as often. The dependable reachability test is your server logs: find requests from the published OpenAI and Anthropic crawler IP ranges and confirm they return 200.

## Put the answer where agents read

Agents use the same homepage and docs pages as human visitors. Agent-specific files (llms.txt, markdown mirrors, JSON-LD) help agents find, fetch, and use those same answers. Getting answers into the initial HTML (SSR or prerendering) is the prerequisite for agents that fetch rather than run JavaScript. A fact that exists only inside a <script> element (JSON-LD, a JSON config, or a hydration payload) is in the HTML but not in the text most agents read, so repeat key facts as visible text. Links matter: agents rarely found pages or files that were not linked from content they had already reached.

Pages agents read:

- AR-READ-03 (SHOULD) Homepage in raw HTML: The front door, with real links to docs, product, and pricing. [STUDIES] Reached in 69% of runs, and it was the agent’s first step in 92% of them; in 59% of the runs that reached it, the next hop was a docs page.
- AR-READ-04 (SHOULD) Docs pages that answer: The pages agents actually read, where most tasks get resolved. Link setup and quickstart pages from the homepage. [STUDIES] Reached in 83% of runs and fetched more than any other page type, about 3.4 docs pages per run.
- AR-READ-11 (SHOULD) Pricing on a first-party page: Plans, free tier, and limits as text in the HTML, not an image, a calculator, or a sales form. [STUDIES] Built from the business’s own pages, 60% of pricing answers were accurate, against 37% from elsewhere, the widest gap of any question type.
- AR-READ-01 (MUST) Fetchable without JavaScript: The answer is in the initial HTML, server-rendered or prerendered, rather than injected by a single-page app after load. [STUDIES] A JavaScript-hidden answer was one of only two changes that broke an agent. The other was blocking it outright.
- AR-READ-05 (SHOULD) Fenced code, language-tagged: Turns snippets into examples an agent can lift and use. [STUDIES] When an agent’s answer included code, 61% of the time it contained lines taken verbatim from a page it fetched on the site.

Files that describe the site. Link each from content agents already visit (footer, <link rel="alternate"> in the head, or a resources section); 86–100% of visits came through links rather than guessed paths [STUDIES]:

- AR-READ-07 (SHOULD) llms.txt: A curated, described index of your key pages. [STUDIES] Reached via links rather than guesses: 86% of its fetches came from a page that pointed to it. One in three agents that read it went on to fetch a page it lists, and 36% drew their final answer from its content.
- AR-READ-06 (SHOULD) .well-known/*: Protocol and metadata discovery. [STUDIES] Reached in about 23% of runs across the sites tested.
- AR-ACT-02 (SHOULD) openapi.json: Machine-readable API contract. [STUDIES] On the sites that ship one, used in 21% of runs.
- AR-READ-08 (SHOULD) JSON-LD: Machine-readable facts and routes in the page. Markdown converters may omit script blocks, so repeat critical facts in visible text.

[STUDIES] Agents requested markdown on about 65% of web fetches; when they specified a format, they chose markdown 96% of the time. Format did not determine task success, so ensure the HTML works before adding a .md mirror, then advertise it with <link rel="alternate" type="text/markdown"> or an HTTP Link: header.

AR-READ-02 (MUST) Return accurate HTTP status codes: 404 for a missing path (never 200), and 429 with a Retry-After header when throttling. AR-READ-09 (MAY) covers the markdown mirror above.

## Let agents act, not just read

To act, an agent needs an operable interface (API, MCP server, WebMCP tools, NLWeb ask endpoint, SDK, or CLI) plus authentication it can complete, and, if the product charges, a way to pay.

- AR-ACT-01 (SHOULD) Auth an agent can get through: The first gate of acting. Best practice: Self-serve key generation, documented scopes. For user-delegated access, OAuth 2.0 with discoverable metadata under /.well-known (RFC 8414, RFC 9728).
- AR-ACT-02 (SHOULD) Public API with a machine-readable contract: The acting surface itself. Best practice: OpenAPI with operation IDs, typed responses and errors, /v1/ versioning, documented rate limits.
- AR-ACT-10 (MUST) An MCP server or a CLI: The interface an agent acts through. Without one, an agent cannot finish the task. Best practice: Ship at least one. An MCP server (AR-ACT-03) serves agents working in chat and IDE hosts; a CLI (AR-ACT-06) serves agents working in a shell. Build the one that fits how agents use the product, and do not ship an empty surface to meet the requirement.
- AR-ACT-03 (SHOULD) MCP server: Standard tool protocol for docs search and actions. Best practice: Serve over Streamable HTTP at /mcp. Keep the docs MCP public. Gate the product MCP with OAuth (protected-resource metadata at /.well-known/oauth-protected-resource per RFC 9728, with PKCE) and list it in the MCP Registry with a server.json. Server cards for pre-connection discovery (AR-ACT-04, MAY) are an emerging convention (SEP-2127, a working-group draft that major hosts have begun to support): ship one if your target hosts read it, but expect the exact path to keep moving.
- AR-ACT-07 (SHOULD) WebMCP: The same tool contract, registered on the page itself. Best practice: If the product has a web UI an agent should operate, register tools on document.modelContext. Feature-detect; the API is a W3C Community Group draft. Complementary to a server MCP: these tools run in the live page with the signed-in session. Only ship tools that map to actions the product actually offers.
- AR-ACT-05 (MAY) MCP Apps: Your UI, rendered inside the agent host. Best practice: If your product has a UI, ship a View (a ui:// resource) so the agent can hand a working surface back to the user.
- AR-ACT-09 (MAY) NLWeb ask endpoint: A lighter alternative to a full tool surface for content-heavy sites. Best practice: Serve /ask, which takes a natural-language question and returns schema.org-shaped results, and expose the same ask tool over MCP. Suits publishers, catalogs, and docs whose value is content rather than operations.
- AR-ACT-06 (SHOULD) SDKs and a CLI: The forms agents act in: code and shell. Best practice: Publish SDKs on npm and PyPI and a CLI; show install and first call in the docs.
- AR-ACT-08 (MAY) Let agents pay: The last gate of acting. Applies only if the product charges. Best practice: Per-call pricing: x402 or MPP, so an agent can settle a 402 and retry without an account. Checkout: UCP or ACP, so a shopping agent can complete a purchase. On Stripe, enable Link on checkout and accept Shared Payment Tokens, so an agent holding a Link Agent Wallet pays with a one-time credential the user approved, without filling a card form. Do not put a CAPTCHA wall in front of a verified agent. [STUDIES] Paying through Link, agents got 92.3% of test checkouts right; 18 of 26 agents got at least 90%.

## Links

- Essentials: https://agentready.org/#essentials
- Guide: https://agentready.org/#guide
- Be reachable and citable: https://agentready.org/#reachable
- Put the answer where agents read: https://agentready.org/#read
- Let agents act: https://agentready.org/#act
- Check your site: https://agentready.org/#beyond
- is-agentic.com: https://is-agentic.com
- ora.ai: https://ora.ai
- ora docs: https://ora.ai/docs
- Mintlify Agent Score: https://www.mintlify.com/score
- AXIS by Netlify: https://axis.run
- WebMCP draft: https://webmachinelearning.github.io/webmcp/
- GitHub: https://github.com/agentready-org/standard
- Spec source: https://github.com/agentready-org/standard/blob/main/spec/spec.md
- Changelog: https://github.com/agentready-org/standard/blob/main/CHANGELOG.md
- Announcement (March 20, 2026): https://ora.ai/blog/agentready-open-standard
- Dataset: https://github.com/agentready-org/standard/tree/main/data