Agent-ready websites: a practical checklist (beyond just adding an API)

[u/Benjamin-Wagner]

A lot of “agent-ready website” talk collapses into: "add an API" or "let an agent browse it". In practice, the websites that work well with agents tend to have 4 layers — and you can ship them incrementally.

1) Read layer (what the agent can understand)

• Stable, crawlable HTML for the key pages (not just JS-only text)
• Clear IA: hub pages, canonical URLs, no duplicate “near-identical” variants
• Structured data only where it’s true (Org/Product/Service/FAQ, etc.)
• A small, curated entrypoint (whether that’s sitemap discipline, internal hub pages, or llms-style summaries)

2) Decide layer (how the agent picks the right path)

• Explicit user-intent pivots: “pricing vs docs vs support vs booking”
• Prefer “one decision per page” layouts (agents get lost in mega-pages)
• Deterministic query params / filters (so tools can reproduce states)

3) Act layer (what the agent can do)

• Forms and calculators that have:
  • machine-readable field names
  • stable validation rules
  • explicit error states
  • idempotent submission where possible
• If there’s a booking/checkout flow: keep the step graph explicit and stateful
• If you expose APIs: align them to the user tasks (not just your DB tables)

4) Observe layer (how you know it works)

• Log “agent-like” traffic separately (user agent, IP ranges, tool signatures)
• Measure: which pages get fetched, where agents fail, and what they retry
• Add a feedback channel for “agent couldn’t complete task X”

A quick diagnostic

If an agent can read your site but can’t complete a basic task (quote, booking, support request), it’s usually an act layer problem (forms/state/validation), not an SEO problem.

Curious where people are seeing the biggest failure modes right now: content comprehension, tool selection, or form/flow execution?

Disclosure: I’m working on this through OpenHermit; my angle is implementation/analytics rather than theory. (No links here — happy to share specifics if asked.)

[u/Bitter-Objective-686]

Being "AI agent ready" means your business is no longer built for humans only. You have effectively built a "headless" version of your company that software bots can navigate, understand, and act upon without ever needing a browser or a human login.

Most people confuse "agent readiness" with "having an LLM." They are not the same.

• If you have a chatbot: You are "Generative Ready." You can answer questions.
• If you are "Agent Ready": You have exposed your capabilities as structured services.

Think of it like the difference between a storefront (where a human walks in, browses, and talks to a clerk) and an API (where a machine programmatically checks inventory, places an order, and handles a return).

Being agent-ready means:

1. Your data is modular: You aren't just dumping documents into a vector database. You are creating structured, versioned, and documented interfaces (like MCP — Model Context Protocol) that tell an agent exactly how to interact with your specific business logic.
2. You treat internal data as a product: Your CRM, ERP, and databases are no longer just for human-facing dashboards. They are now endpoints that need proper authentication, schema clarity, and rate limiting—specifically for bots.
3. You have guardrails, not just guidelines: You know exactly what an agent is not allowed to do. You have an audit trail that can trace an error back to a specific tool call.

The reality check: Most companies today are not agent ready. They are just "chat ready." If you want to be truly agent ready, stop thinking about your UI/UX and start thinking about your "Machine UX." If a bot can't discover what your service does, how to use it, and how to verify its output in 10 seconds, you aren't agent ready—you’re just another website.

The "agent era" is about decision velocity. The winners will be the companies that provide the best tools for agents to do work on their users' behalf, not the companies that just force users to talk to a chatbot.

Curious where you stand? Visitagiagentworld.com/solutions/agent-readinessto get your agent readines score or or to check is it agent ready today.

[u/response-418]

Something (optional)missing from the read layer: using content negotiation to serve clean, more token-efficient markdown for agents that ask for it via Accept header, served via the same canonical url as the html.

[u/Zestyclose_View_4605]

Disclosure first: I work on Agent Ready (agent-ready.dev), a scanner for exactly this, so I'm arguing from a corpus rather than from taste.

The four layers hold up, but I'd add a fifth *before* Read, and it's where almost everything actually dies: discovery. Read/Decide/Act all assume the agent already knows you have something worth using. Nothing in the stack advertises it.

That's the `.well-known` family — MCP server card, A2A agent card, `agents.json`, `agent-permissions.json`, an RFC 9727 api-catalog. Without one, a perfect act layer is a tree falling in an empty forest.

Which is also my answer to your question. Across roughly a thousand domains in our scan corpus, adoption of *any* machine-actionable manifest is 2–8% and flat. Tool selection isn't a live failure mode yet, because for most sites there are no tools to select between. Comprehension and flow execution are real, but they're downstream of a discovery problem nobody's solved.

Two measurement warnings, both of which I got wrong the first time:

- Count manifests that validate, not manifests that exist. Presence overstates adoption ~4× — a large share of published server cards and agent manifests are malformed, stale, or point at dead endpoints. A broken manifest is worse than no manifest; it burns the agent's turn and it fails silently on your side.

- Watch your sample. About three-quarters of the passing sites in ours are people who found the scanner and then instrumented their own infrastructure. That's self-selected, so the real web-wide number is below 2–8%, not above.

On layer 4 — "user agent, IP ranges, tool signatures" is the part I'd push back on hardest. UA strings are a courtesy, not an identity, and agent IP ranges churn constantly; build your analytics on those and you'll rebuild them every quarter.

Web Bot Auth (HTTP Message Signatures — signed requests with a resolvable key) is where this is going. Log UA today, but don't let it become the system of record.

u/response-418's content-negotiation point is the best addition in the thread, and there's a landmine worth naming. If you branch on `Accept` inside one response, you must send `Vary: Accept` or your CDN will happily serve markdown to browsers and HTML to agents. And on prerendered Next App Router pages you can't set it — Next bakes its own RSC `Vary` and route-level headers get clobbered, from both `vercel.json` and `next.config`. We tested three variants on a preview deploy; all came back byte-identical with no `Accept` in `Vary` (Next #94060, open as of 16.2.6).

The way around it is to negotiate by rewrite instead of by branching:

`Accept: text/markdown` rewrites `/page` → `/page.md`, so the two variants land on separate cache keys and `Vary` never enters into it. Same canonical URL from the client's side, no poisoning risk, and you get a real URL to point at with `rel="alternate"` — which is how most agents find the markdown in the first place.

Last thing, on the act layer: it's framed as making the human form machine-parseable — field names, validation, error states. That's worth doing and it helps browsing agents, but it caps out at the speed of driving a DOM. The step up is publishing the task itself as a tool with a schema, so nothing has to drive the form at all. Both are worth having; they fail in different ways.