# Agent-Ready Websites & WebMCP

> Audit sites 0-100 with is-agentic, publish /llms.txt maps and Accept: text/markdown content negotiation, expose browser-native WebMCP tools, and enforce human-in-the-loop safety gates.

- **Topic:** [AI Visibility & Work](https://twelve-pages.com/topics/visibility-and-workflows)
- **Web Edition:** https://twelve-pages.com/guides/visibility-and-workflows/agent-ready-websites-and-webmcp
- **Printable 12-Page PDF:** https://twelve-pages.com/guides/visibility-and-workflows/agent-ready-websites-and-webmcp/Agent_Ready_Websites_is_agentic_and_WebMCP_Complete_How_To_Guide.pdf (2.4 MB)
- **Version:** v1.0 (Updated 2026-11)
- **Reading Time:** 12 minutes (12 pages)
- **Authorship:** AI-authored under human editorial direction & print QA (https://twelve-pages.com/editorial)
- **Tags:** is-agentic Audits · llms.txt & llms-full.txt · Accept: text/markdown · Browser WebMCP Tools · RFC 9457 Errors · Human Checkout Gates

## Core Takeaways

- **0–100 Readiness Audits:** Evaluate Essential, Recommended, and Bonus checks via npx is-agentic, the public REST API, or MCP.
- **Zero-Bloat Markdown:** Publish /llms.txt maps and Accept: text/markdown negotiation to cut HTML token waste by over 80%.
- **Native Browser WebMCP:** Turn HTML forms and frontend functions into typed client-side tools guarded by explicit human confirmation.

## 1. The Dual-Stack Web: Human Interfaces Meet Agent Protocols

Every modern website now serves two very different visitors: people browsing rich visual layouts, and AI agents that need clean Markdown, structured error contracts, and typed browser tools instead of brittle screen scraping.

### Quick Navigation

#### 1 Discovery Layer

Clear `robots.txt` rules, XML sitemaps, JSON-LD metadata, and `/llms.txt` tell agents what your site offers before they fetch a single page.

#### 2 Content Channel

Server-rendered HTML and `Accept: text/markdown` negotiation strip navigation chrome, scripts, and CSS clutter down to pure facts.

#### 3 Browser Tool Layer

WebMCP lets pages expose deterministic actions via HTML forms or `registerTool()`, protected by human approval prompts.

### Architecture: Request Routing Across Discovery, Markdown, Tools & Errors

## 2. Setup, is-agentic Audit Tiers & Protocol Matrix

You can measure any public website in seconds with zero API keys or signups, then choose the right mix of Markdown endpoints and WebMCP tools for your architecture.

### Quickstart: Run Audits & Enable Experimental Browser WebMCP

```
<code># 1. Run an instant CLI readiness scan (returns stored report or triggers a fresh scan)
npx is-agentic https://example.com --json

# 2. Query an existing snapshot via the public read-only API or install the agent skill
curl -s "https://is-agentic.com/api/v1/report?url=https%3A%2F%2Fexample.com"
npx skills add vercel-labs/is-agentic

# 3. Enable WebMCP in Chrome 146+ for local tool testing:
# Open chrome://flags/#enable-webmcp-testing -> Enabled -> Relaunch</code>
```

### How `is-agentic` Scores Every Website (0 to 100)

#### E Essential (80-Pt Pool)

Core survival checks: clean HTTP status codes, server-rendered content without JS (`content-no-js`), permissive `robots.txt` rules, and clear page headings.

#### R Recommended (20-Pt Pool)

Efficiency and discoverability signals: `/llms.txt` directory, `Accept: text/markdown` negotiation, XML sitemaps, and Schema.org `JSON-LD` metadata.

#### B Bonus (Up to +5 Pts)

Additive `positive_signals` for emerging standards like WebMCP, public MCP servers, or commerce feeds. Missing bonus items never reduce your score.

### Agentic Web Protocol & Standard Matrix

| Protocol / Standard | Surface & Entry Point | Primary Benefit | Best Use Case |
| --- | --- | --- | --- |
| **is-agentic Audit** | `npx is-agentic`, REST API, `/mcp` | 0–100 score + evidence-backed fixes | Baseline site benchmarking, CI gates, and post-deploy verification. |
| **`/llms.txt` & `/llms-full.txt`** | Root Markdown files | Curated link index & bulk docs dump | Documentation portals, SDK references, pricing and policy hubs. |
| **`Accept: text/markdown`** | HTTP header content negotiation | Same URL, 80%+ fewer tokens | Blogs, product pages, knowledge bases shared by humans and bots. |
| **Declarative WebMCP** | HTML `<form toolname>` attributes | Zero-JS schema synthesis | Booking forms, search filters, support tickets, lead capture flows. |
| **Imperative WebMCP** | `navigator.modelContext.registerTool` | Dynamic SPA state & async handlers | Complex configurators, live carts, multi-step dashboards, user gates. |
| **RFC 9457 Errors** | `application/problem+json` | Machine-actionable recovery hints | Rate limits (429), auth errors, missing scans (404), validation faults. |

## 3. Annotated Workspace: Is-Agentic Report & WebMCP Inspector

Diagnosing agent readiness means reading both sides of the stack at once: server-side audit evidence from is-agentic and live tool registrations inside the browser’s modelContext inspector.

Agentic Readiness Console • acme-store.dev [WebMCP Enabled]

Snapshot Age: 14m (6h Cache TTL)

**Score:** 88/100 • Ess: `72/80` • Rec: `12/20` • Bonus: `+4`

**View:** `App & Docs` • `scanned_at: 2026-11-16T14:20Z`

PublicScanReport Issues & Tier Breakdown

[PASS] content-no-js (essential) • SSR HTML 42KB

[PASS] llms-txt (recommended) • 14 curated links

[PARTIAL] accept-markdown • Missing Vary: Accept

[FAILED] problem-json (essential) • 404 returns HTML

Edge Content Negotiation Trace

GET /docs/webhooks • Accept: text/markdown

200 OK • Content-Type: text/markdown; charset=utf-8

x-markdown-tokens: 580 (vs 6,420 HTML tokens)

Registered Browser WebMCP Tools (3 Active)

`search_catalog` [readOnlyHint: true]

`book_demo_slot` [Declarative Form]

`place_order` [consequentialHint: true]

Schema validated via navigator.modelContextTesting

Declarative Form SubmitEvent Bridge

Event: submit • `e.agentInvoked: true`

e.respondWith({ status: 'confirmed', slotId: 'D-19' })

**Human Gate:** `client.requestUserInteraction()` paused `place_order` • [Approve Modal Open]

[Rescan] • [Export JSON]

### Control Legend & Inspection Points

**Composite Score & Pools:** Separates core 80-point Essential health from 20-point Recommended enhancements and optional +5 Bonus signals.

**Evidence-Backed Issues List:** Pinpoints failing or partial checks with exact observed HTML/header evidence and targeted fix guidance.

**Markdown Negotiation Trace:** Confirms clean `text/markdown` responses and tracks dramatic token reductions via `x-markdown-tokens`.

**Contextual Lens & Timestamp:** Switches audit emphasis by site type while tracking the immutable `scanned_at` timestamp against the 6-hour cache.

**Live WebMCP Tool Registry:** Lists all tools registered on the active page alongside safety hints like `readOnlyHint` and `consequentialHint`.

**Declarative SubmitEvent Hook:** Captures agent-submitted HTML forms via `e.agentInvoked` and returns structured confirmation with `e.respondWith()`.

**Human Confirmation Gate:** Suspends consequential checkout or write operations using `requestUserInteraction()` until the user explicitly clicks Approve.

#### Two Complementary Perspectives: Crawl Health vs. Live Interaction

`is-agentic` evaluates what autonomous agents see before logging in—clean HTML, `/llms.txt` discovery, and Markdown headers—while Chrome's WebMCP inspector verifies what authenticated agents can safely invoke inside an open browser tab.

## 4. Auditing & Fixing Sites with is-agentic CLI, API & MCP

Every check in is-agentic is grounded in live HTTP, HTML, and protocol evidence. Pick the interface that matches your environment—command line, REST API, or native MCP tool calls.

#### 1 Official CLI

Run `npx is-agentic <domain> --json` or `bunx is-agentic`. Returns a cached report at once, or starts a scan and waits if none exists.

#### 2 Read-Only REST API

Call `GET /api/v1/report?url=...`. Never starts a scan; returns `404` if missing. Supports `Accept: text/markdown` for compact text.

#### 3 Streamable MCP

Connect AI hosts to `https://is-agentic.com/mcp` for `is_agentic_get_report`, `_get_methodology`, and `_get_developer_docs`.

### Anatomy of a `PublicScanReport` JSON Response

```
<code>{
  "score": 84,
  "score_label": null,
  "report_url": "https://is-agentic.com/report/example.com",
  "scanned_at": "2026-11-16T14:20:00.000Z",
  "score_breakdown": {
    "essential":   { "earned": 70, "available": 80, "passing": 7, "total": 8 },
    "recommended": { "earned": 14, "available": 20, "passing": 4, "total": 6 },
    "bonus":       { "points": 3, "positive_signals": ["llms-full-txt", "webmcp"] }
  },
  "issues": [
    {
      "id": "accept-markdown",
      "name": "Markdown content negotiation",
      "tier": "recommended",
      "result": "failed",
      "details": "GET / returned text/html even when Accept: text/markdown was sent.",
      "recommendation": "Serve text/markdown with Vary: Accept for documentation and article routes."
    }
  ]
}</code>
```

#### Prioritized Remediation Formula (Never Chase Bonus Points First)

1. Essential Failures -> 2. Essential Partials -> 3. Recommended Checks -> 4. Deploy & Verify scanned_at

Always apply the specific fix in each issue's `recommendation` field because it is generated directly from observed scan evidence. Ignore missing `bonus` signals until every Essential and Recommended item passes.

#### A Site-Type Contextual Views

Reports adapt to **Docs & Content**, **Business**, **App**, or **Commerce** views. Checks that do not apply (such as API schemas on a personal essay site) are excluded cleanly instead of hurting your score.

#### B Handling `score: null` Targets

When an endpoint cannot be audited publicly—such as an OAuth-protected MCP server—`score` is `null` and `score_label` explains why. Remember: paths and query strings produce separate reports.

## 5. Zero-Bloat Discovery: /llms.txt & Accept: text/markdown

Raw HTML pages wrap a few paragraphs of useful knowledge inside megabytes of navigation menus, React hydration bundles, and cookie banners. Clean Markdown files and HTTP content negotiation eliminate that waste.

*[Illustration: Visualizing raw HTML web pages streamlining into clean glowing Markdown channels for AI ingestion]*

**Figure 6.1 • Token-Light Content Negotiation:** Heavy visual web pages route through an edge negotiator when an agent sends `Accept: text/markdown`, delivering clean, structured Markdown at the exact same canonical URL.

#### 1 `/llms.txt` & `/llms-full.txt`

Place `/llms.txt` at your domain root using four simple Markdown elements: one `# H1` site title, a `> blockquote` summary, `## Section` headings with `- [Title](url): Description` links, and an optional `## Optional` section for secondary reading.

Pair it with `/llms-full.txt` when you want agents to ingest an entire documentation corpus in a single fetch.

#### 2 `Accept: text/markdown` Headers

Keep URLs identical for humans and bots. When a client sends `Accept: text/markdown`, respond with `Content-Type: text/markdown; charset=utf-8`, include `Vary: Accept` so CDNs cache both variants separately, and optionally expose `x-markdown-tokens`.

Advertise alternate links in HTML via `<link rel="alternate" type="text/markdown">`.

### Next.js App Router Middleware: Serving Clean Markdown on the Same URL

```
<code>import { NextRequest, NextResponse } from 'next/server';

export function middleware(req: NextRequest) {
  const accept = req.headers.get('accept') || '';
  if (accept.includes('text/markdown')) {
    const target = req.nextUrl.clone();
    target.pathname = '/api/markdown' + target.pathname;
    const res = NextResponse.rewrite(target);
    res.headers.set('Vary', 'Accept');
    return res;
  }
  return NextResponse.next();
}</code>
```

## 6. Browser WebMCP: Turning Pages into Native Agent Tools

Developed in the W3C Web Machine Learning Community Group and available behind chrome://flags/#enable-webmcp-testing in Chrome 146+, WebMCP lets websites publish structured tools directly inside the user’s browser session.

#### 1 Declarative HTML Form Attributes

Turn standard forms into typed tools with zero JavaScript boilerplate using `toolname`, `tooldescription`, optional `toolautosubmit`, and `toolparamdescription` on fields. The browser converts HTML constraints (`required`, `type="date"`, `<select>`) into JSON Schema and sets `e.agentInvoked` on submit.

#### 2 Imperative `modelContext` API

Register dynamic JavaScript tools with `navigator.modelContext.registerTool()` (note: newer W3C drafts also align on `document.modelContext`). Manage SPA view changes cleanly with `unregisterTool()`, `provideContext()`, and `clearContext()`, and label safe reads with `readOnlyHint: true`.

#### WebMCP Tool Registration Contract Anatomy

toolname / name + tooldescription + JSON inputSchema + execute(input, client) + readOnlyHint / consequentialHint

### Pattern A: Declarative HTML Form with Agent Response Hook

```
<code><form id="demo-form" toolname="book_demo" tooldescription="Reserve a product walkthrough slot" toolautosubmit>
  <input name="email" type="email" required toolparamdescription="Work email for calendar invite" />
  <input name="date" type="date" required toolparamdescription="Preferred date in YYYY-MM-DD" />
  <button type="submit">Book Demo</button>
</form>
<script>
  document.getElementById('demo-form').addEventListener('submit', (e) => {
    e.preventDefault();
    const booking = saveDemoSlot(new FormData(e.target));
    if (e.agentInvoked && typeof e.respondWith === 'function') {
      e.respondWith(Promise.resolve({ content: [{ type: 'text', text: JSON.stringify(booking) }] }));
    }
  });
</script></code>
```

### Pattern B: Imperative Read-Only Lookup via `navigator.modelContext.registerTool`

```
<code>if ('modelContext' in navigator) {
  navigator.modelContext.registerTool({
    name: 'search_docs',
    description: 'Search product documentation pages by keyword and return matching snippets.',
    inputSchema: {
      type: 'object',
      properties: { query: { type: 'string', description: 'Search keyword or error code' } },
      required: ['query']
    },
    readOnlyHint: true,
    execute: async ({ query }) => ({
      content: [{ type: 'text', text: JSON.stringify(await runLocalDocSearch(query)) }]
    })
  });
}</code>
```

## 7. Machine-Readable Errors: RFC 9457 & Agent Recovery

Returning a 40KB styled HTML error page for a rate limit or missing route forces an LLM to burn tokens guessing what went wrong. RFC 9457 application/problem+json turns failures into deterministic instructions.

*[Illustration: Luminous amber protective dome enclosing a central core amid an organized grid of emerald hex nodes]*

**Figure 8.1 • Deterministic Error & Protocol Contracts:** Instead of dumping unparsed HTML error pages into an agent context window, RFC 9457 gateways wrap failures in a golden protective schema carrying a stable `code` and an explicit `resolution` hint.

### RFC 9457 `application/problem+json` Response Contract

```
<code>HTTP/1.1 429 Too Many Requests
Content-Type: application/problem+json
Retry-After: 30

{
  "type": "https://is-agentic.com/errors/rate-limit-exceeded",
  "title": "Rate limit exceeded",
  "status": 429,
  "detail": "Exceeded 120 requests per 60 seconds from this IP address.",
  "code": "rate_limit_exceeded",
  "resolution": "Honor the Retry-After header and wait 30 seconds before retrying."
}</code>
```

| Stable `code` | HTTP | Meaning | Automated Agent Recovery Action |
| --- | --- | --- | --- |
| `invalid_url` | `400` | Target is malformed or private | Validate scheme and pass a public `http://` or `https://` URL. |
| `report_not_found` | `404` | No cached scan exists yet | Run `npx is-agentic <url> --json` or open `/scan/<target>`. |
| `rate_limit_exceeded` | `429` | Burst exceeded 120 req / 60s | Read the HTTP `Retry-After` header and sleep before retrying. |
| `report_temporarily_unavailable` | `503` | Transient backend hiccup | Retry briefly with backoff; do not trigger duplicate scans. |

## 8. Five Copy-Paste Recipes for Agent-Ready Websites

Drop these battle-tested templates into documentation sites, SaaS applications, and storefronts to raise your is-agentic score and unlock reliable agent interactions.

| Recipe & Goal | Stack | Implementation Pattern | Verification Step |
| --- | --- | --- | --- |
| **1. CI Readiness Gate** Block regressions on PR deploy | GitHub Actions + CLI | `npx is-agentic $URL --json` parsed with `jq` or Node threshold check | Confirm `essential.passing == total` |
| **2. Curated `/llms.txt`** Zero-noise documentation map | Static file or build script | Single `# H1`, `> blockquote`, grouped `- [Title](url): desc` links | Validate `200 OK` & valid Markdown links |
| **3. Edge Markdown Switch** Token-light page delivery | Next.js Middleware or CDN | Inspect `Accept: text/markdown`, rewrite route, set `Vary: Accept` | Test `curl -H "Accept: text/markdown"` |
| **4. Zero-JS Booking Tool** Agent-callable HTML forms | HTML5 + WebMCP attributes | Add `toolname`, `tooldescription`, `toolparamdescription` + `e.respondWith` | Verify schema in Chrome WebMCP flag |
| **5. SPA Tool Lifecycle** Sync tools with route views | React / Vue + WebMCP | Call `registerTool` on mount and `unregisterTool(name)` on cleanup | Run `clearContext()` on sign-out |

### Copy-Paste `/llms.txt` Root Template

```
<code># Acme Cloud Platform

> Developer documentation, REST endpoints, and billing guides for Acme Cloud.

## Core Guides
- [Quickstart Guide](https://acme.example/docs/quickstart.md): Deploy your first service in three minutes.
- [Authentication API](https://acme.example/docs/auth.md): Scoped API tokens, OAuth flows, and webhook signing.

## Optional
- [Complete Corpus Export](https://acme.example/llms-full.txt): Single-file documentation bundle for bulk indexing.</code>
```

### Copy-Paste CI/CD Readiness Gate (Shell + Node One-Liner)

```
<code>REPORT=$(npx is-agentic https://staging.acme.example --json)
node -e "const r=JSON.parse(process.argv[1]); if (r.score !== null && r.score < 85) process.exit(1);" "$REPORT"</code>
```

## 9. Security Boundaries, Rate Limits & Human Confirmation Gates

Because browser WebMCP tools run directly inside an authenticated user’s session with live cookies, high-stakes actions like purchases or account deletions must never execute silently.

#### 1 120 req / 60s Rate Limit

`is-agentic.com` enforces 120 requests per IP per 60 seconds. Cache JSON results locally in CI and obey HTTP `Retry-After` headers on `429` responses.

#### 2 6-Hour Snapshot Cache

Reports are immutable snapshots that refresh only when a visit finds them older than 6 hours. Compare `scanned_at` after deploys or trigger manual Rescan.

#### 3 Auth Surface Hygiene

Private MCP servers yield `score: null`. Keep internal admin dashboards out of `/llms.txt` and `sitemap.xml` while allowing public docs.

### Human-in-the-Loop Checkout Gate Using `client.requestUserInteraction()`

```
<code>navigator.modelContext.registerTool({
  name: 'place_order',
  description: 'Submit the active shopping cart for final checkout after user confirmation.',
  inputSchema: {
    type: 'object',
    properties: { shippingMethod: { type: 'string', enum: ['standard', 'express'] } },
    required: ['shippingMethod']
  },
  consequentialHint: true,
  execute: async ({ shippingMethod }, client) => {
    const approved = await client.requestUserInteraction(async () => {
      return window.confirm('Confirm checkout using ' + shippingMethod + ' shipping?');
    });
    if (!approved) {
      throw new Error('Checkout cancelled by user.');
    }
    const res = await fetch('/api/checkout', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ shippingMethod })
    });
    const order = await res.json();
    return { content: [{ type: 'text', text: 'Confirmed Order ID: ' + order.orderId }] };
  }
});</code>
```

#### Proposal-Only Tool Design & Backend Authorization

Treat every mutating WebMCP tool as a proposal rather than an authoritative command. Display exact dollar totals, recipients, and shipping terms inside your confirmation modal, and verify session tokens, CSRF origins, and cryptographic bot-auth signatures independently on the backend API before committing any write.

## 10. Cross-Tool Synergy: From Source Docs to Verified Browser Actions

When static discovery files, edge Markdown negotiation, automated is-agentic audits, and browser WebMCP tools work in concert, agents spend fewer tokens reading and never break on brittle UI selectors.

#### 1 Coding Agents + `is-agentic` MCP Loop

Wire `https://is-agentic.com/mcp` into Claude Code, Cursor, or Antigravity alongside `npx skills add vercel-labs/is-agentic`. Your agent pulls failing Essential checks via `is_agentic_get_report`, edits your Next.js middleware or `/llms.txt` generator, deploys, and verifies the improvement.

#### 2 Markdown Context + WebMCP Execution

First, agents read pricing, return policies, and API guides cleanly through `/llms.txt` and `Accept: text/markdown`. Once the user visits the interactive page in Chrome or Edge, WebMCP takes over to invoke search filters, cart updates, and gated checkouts without fragile DOM clicking.

#### One Canonical URL, Zero Duplicate Logic

Because Markdown negotiation reuses your human URLs and WebMCP wraps existing form handlers and frontend API calls, you never maintain a shadow bot website. Every improvement in structured clarity benefits search crawlers, coding assistants, and assistive browser agents simultaneously.

### Connecting Your IDE or Agent Host via Streamable HTTP MCP

```
<code>{
  "mcpServers": {
    "is-agentic": {
      "type": "http",
      "url": "https://is-agentic.com/mcp"
    }
  }
}</code>
```

## 11. Troubleshooting Clinic & 6-Point Pre-Flight Checklist

Fast, evidence-backed fixes for the seven most common agentic web integration pitfalls, paired with a pre-flight readiness checklist before you ship to production.

| Symptom & Error | Root Cause | 10-Second Production Fix |
| --- | --- | --- |
| **Old score after deploying fix** | Cached snapshot (< 6 hours old) | Compare `scanned_at`; click **Rescan** on `report_url` for an immediate re-run. |
| **`404 report_not_found`** | Read-only REST API never scans | Run `npx is-agentic <url> --json` or open `/scan/<target>` first. |
| **`score: null` on MCP target** | Endpoint requires authentication | Check `score_label`; publish public docs or unauthenticated metadata card. |
| **CDN shows Markdown to humans** | Missing `Vary: Accept` header | Set `Vary: Accept` on every negotiated response and purge edge cache. |
| **`content-no-js` Essential fail** | Client-only CSR renders blank shell | Enable SSR or static prerendering (SSG) for core content routes. |
| **`modelContext` is undefined** | Missing Chrome flag or HTTPS | Enable `#enable-webmcp-testing`, serve over HTTPS, and guard with `'modelContext' in navigator`. |
| **`429 rate_limit_exceeded`** | Exceeded 120 req / 60s IP limit | Read the `Retry-After` header, back off, and cache reports across CI jobs. |

### 6-Point Pre-Flight Agent-Readiness Checklist

**&square; 1. Essential Tier Green:** Run `npx is-agentic <domain> --json` and confirm zero failed or partial checks inside `score_breakdown.essential`.

**&square; 2. Curated `/llms.txt` Live:** Confirm `GET /llms.txt` returns HTTP `200` with a crisp `# H1`, blockquote summary, and valid Markdown links.

**&square; 3. Safe Cache Negotiation:** Test `Accept: text/markdown` and `Accept: text/html` back-to-back to verify `Vary: Accept` isolates both caches.

**&square; 4. RFC 9457 Error Contracts:** Trigger 400, 404, and 429 paths with an agent header and confirm clean `application/problem+json` payloads.

**&square; 5. WebMCP Lifecycle Clean:** Verify declarative forms return `e.respondWith()` and SPA views call `unregisterTool()` on unmount.

**&square; 6. Human Checkout Gate Tested:** Trigger consequential actions and verify `requestUserInteraction()` blocks execution until explicit user approval.

---

## Edition Note & Attribution

This 12-page handbook, its topic structure, and its illustrations were generated with AI under human editorial direction and print-layout QA. Verify commands, pricing tiers, and security settings against official vendor documentation before production use.

- AI & Editorial Disclosure: https://twelve-pages.com/editorial
- Terms & Licensing: https://twelve-pages.com/terms
- Privacy & Cookie Notice: https://twelve-pages.com/privacy
