1. The Dual-Stack Web: Human Interfaces Meet Agent Protocols
Every modern website now serves two very different visitors: people browsing rich visual layouts, and AI agents that need clean Markdown, structured error contracts, and typed browser tools instead of brittle screen scraping.
Quick Navigation
Clear robots.txt rules, XML sitemaps, JSON-LD metadata, and /llms.txt tell agents what your site offers before they fetch a single page.
Server-rendered HTML and Accept: text/markdown negotiation strip navigation chrome, scripts, and CSS clutter down to pure facts.
WebMCP lets pages expose deterministic actions via HTML forms or registerTool(), protected by human approval prompts.
Architecture: Request Routing Across Discovery, Markdown, Tools & Errors
2. Setup, is-agentic Audit Tiers & Protocol Matrix
You can measure any public website in seconds with zero API keys or signups, then choose the right mix of Markdown endpoints and WebMCP tools for your architecture.
Quickstart: Run Audits & Enable Experimental Browser WebMCP
# 1. Run an instant CLI readiness scan (returns stored report or triggers a fresh scan)
npx is-agentic https://example.com --json
# 2. Query an existing snapshot via the public read-only API or install the agent skill
curl -s "https://is-agentic.com/api/v1/report?url=https%3A%2F%2Fexample.com"
npx skills add vercel-labs/is-agentic
# 3. Enable WebMCP in Chrome 146+ for local tool testing:
# Open chrome://flags/#enable-webmcp-testing -> Enabled -> RelaunchHow is-agentic Scores Every Website (0 to 100)
Core survival checks: clean HTTP status codes, server-rendered content without JS (content-no-js), permissive robots.txt rules, and clear page headings.
Efficiency and discoverability signals: /llms.txt directory, Accept: text/markdown negotiation, XML sitemaps, and Schema.org JSON-LD metadata.
Additive positive_signals for emerging standards like WebMCP, public MCP servers, or commerce feeds. Missing bonus items never reduce your score.
Agentic Web Protocol & Standard Matrix
| Protocol / Standard | Surface & Entry Point | Primary Benefit | Best Use Case |
|---|---|---|---|
| is-agentic Audit | npx is-agentic, REST API, /mcp |
0–100 score + evidence-backed fixes | Baseline site benchmarking, CI gates, and post-deploy verification. |
/llms.txt & /llms-full.txt |
Root Markdown files | Curated link index & bulk docs dump | Documentation portals, SDK references, pricing and policy hubs. |
Accept: text/markdown |
HTTP header content negotiation | Same URL, 80%+ fewer tokens | Blogs, product pages, knowledge bases shared by humans and bots. |
| Declarative WebMCP | HTML <form toolname> attributes |
Zero-JS schema synthesis | Booking forms, search filters, support tickets, lead capture flows. |
| Imperative WebMCP | navigator.modelContext.registerTool |
Dynamic SPA state & async handlers | Complex configurators, live carts, multi-step dashboards, user gates. |
| RFC 9457 Errors | application/problem+json |
Machine-actionable recovery hints | Rate limits (429), auth errors, missing scans (404), validation faults. |
3. Annotated Workspace: Is-Agentic Report & WebMCP Inspector
Diagnosing agent readiness means reading both sides of the stack at once: server-side audit evidence from is-agentic and live tool registrations inside the browser’s modelContext inspector.
72/80 • Rec: 12/20 • Bonus: +4
App & Docs • scanned_at: 2026-11-16T14:20Z
search_catalog [readOnlyHint: true]book_demo_slot [Declarative Form]place_order [consequentialHint: true]e.agentInvoked: trueclient.requestUserInteraction() paused place_order • [Approve Modal Open]
Control Legend & Inspection Points
1 Composite Score & Pools: Separates core 80-point Essential health from 20-point Recommended enhancements and optional +5 Bonus signals.
2 Evidence-Backed Issues List: Pinpoints failing or partial checks with exact observed HTML/header evidence and targeted fix guidance.
3 Markdown Negotiation Trace: Confirms clean text/markdown responses and tracks dramatic token reductions via x-markdown-tokens.
4 Contextual Lens & Timestamp: Switches audit emphasis by site type while tracking the immutable scanned_at timestamp against the 6-hour cache.
5 Live WebMCP Tool Registry: Lists all tools registered on the active page alongside safety hints like readOnlyHint and consequentialHint.
6 Declarative SubmitEvent Hook: Captures agent-submitted HTML forms via e.agentInvoked and returns structured confirmation with e.respondWith().
7 Human Confirmation Gate: Suspends consequential checkout or write operations using requestUserInteraction() until the user explicitly clicks Approve.
is-agentic evaluates what autonomous agents see before logging in—clean HTML, /llms.txt discovery, and Markdown headers—while Chrome’s WebMCP inspector verifies what authenticated agents can safely invoke inside an open browser tab.
4. Auditing & Fixing Sites with is-agentic CLI, API & MCP
Every check in is-agentic is grounded in live HTTP, HTML, and protocol evidence. Pick the interface that matches your environment—command line, REST API, or native MCP tool calls.
Run npx is-agentic <domain> --json or bunx is-agentic. Returns a cached report at once, or starts a scan and waits if none exists.
Call GET /api/v1/report?url=.... Never starts a scan; returns 404 if missing. Supports Accept: text/markdown for compact text.
Connect AI hosts to https://is-agentic.com/mcp for is_agentic_get_report, _get_methodology, and _get_developer_docs.
Anatomy of a PublicScanReport JSON Response
{
"score": 84,
"score_label": null,
"report_url": "https://is-agentic.com/report/example.com",
"scanned_at": "2026-11-16T14:20:00.000Z",
"score_breakdown": {
"essential": { "earned": 70, "available": 80, "passing": 7, "total": 8 },
"recommended": { "earned": 14, "available": 20, "passing": 4, "total": 6 },
"bonus": { "points": 3, "positive_signals": ["llms-full-txt", "webmcp"] }
},
"issues": [
{
"id": "accept-markdown",
"name": "Markdown content negotiation",
"tier": "recommended",
"result": "failed",
"details": "GET / returned text/html even when Accept: text/markdown was sent.",
"recommendation": "Serve text/markdown with Vary: Accept for documentation and article routes."
}
]
}Always apply the specific fix in each issue’s recommendation field because it is generated directly from observed scan evidence. Ignore missing bonus signals until every Essential and Recommended item passes.
Reports adapt to Docs & Content, Business, App, or Commerce views. Checks that do not apply (such as API schemas on a personal essay site) are excluded cleanly instead of hurting your score.
score: null TargetsWhen an endpoint cannot be audited publicly—such as an OAuth-protected MCP server—score is null and score_label explains why. Remember: paths and query strings produce separate reports.
5. Zero-Bloat Discovery: /llms.txt & Accept: text/markdown
Raw HTML pages wrap a few paragraphs of useful knowledge inside megabytes of navigation menus, React hydration bundles, and cookie banners. Clean Markdown files and HTTP content negotiation eliminate that waste.
Accept: text/markdown, delivering clean, structured Markdown at the exact same canonical URL.
/llms.txt & /llms-full.txt
Place /llms.txt at your domain root using four simple Markdown elements: one # H1 site title, a > blockquote summary, ## Section headings with - [Title](url): Description links, and an optional ## Optional section for secondary reading.
Pair it with /llms-full.txt when you want agents to ingest an entire documentation corpus in a single fetch.
Accept: text/markdown Headers
Keep URLs identical for humans and bots. When a client sends Accept: text/markdown, respond with Content-Type: text/markdown; charset=utf-8, include Vary: Accept so CDNs cache both variants separately, and optionally expose x-markdown-tokens.
Advertise alternate links in HTML via <link rel="alternate" type="text/markdown">.
Next.js App Router Middleware: Serving Clean Markdown on the Same URL
import { NextRequest, NextResponse } from 'next/server';
export function middleware(req: NextRequest) {
const accept = req.headers.get('accept') || '';
if (accept.includes('text/markdown')) {
const target = req.nextUrl.clone();
target.pathname = '/api/markdown' + target.pathname;
const res = NextResponse.rewrite(target);
res.headers.set('Vary', 'Accept');
return res;
}
return NextResponse.next();
}6. Browser WebMCP: Turning Pages into Native Agent Tools
Developed in the W3C Web Machine Learning Community Group and available behind chrome://flags/#enable-webmcp-testing in Chrome 146+, WebMCP lets websites publish structured tools directly inside the user’s browser session.
Turn standard forms into typed tools with zero JavaScript boilerplate using toolname, tooldescription, optional toolautosubmit, and toolparamdescription on fields. The browser converts HTML constraints (required, type="date", <select>) into JSON Schema and sets e.agentInvoked on submit.
modelContext API
Register dynamic JavaScript tools with navigator.modelContext.registerTool() (note: newer W3C drafts also align on document.modelContext). Manage SPA view changes cleanly with unregisterTool(), provideContext(), and clearContext(), and label safe reads with readOnlyHint: true.
Pattern A: Declarative HTML Form with Agent Response Hook
<form id="demo-form" toolname="book_demo" tooldescription="Reserve a product walkthrough slot" toolautosubmit>
<input name="email" type="email" required toolparamdescription="Work email for calendar invite" />
<input name="date" type="date" required toolparamdescription="Preferred date in YYYY-MM-DD" />
<button type="submit">Book Demo</button>
</form>
<script>
document.getElementById('demo-form').addEventListener('submit', (e) => {
e.preventDefault();
const booking = saveDemoSlot(new FormData(e.target));
if (e.agentInvoked && typeof e.respondWith === 'function') {
e.respondWith(Promise.resolve({ content: [{ type: 'text', text: JSON.stringify(booking) }] }));
}
});
</script>Pattern B: Imperative Read-Only Lookup via navigator.modelContext.registerTool
if ('modelContext' in navigator) {
navigator.modelContext.registerTool({
name: 'search_docs',
description: 'Search product documentation pages by keyword and return matching snippets.',
inputSchema: {
type: 'object',
properties: { query: { type: 'string', description: 'Search keyword or error code' } },
required: ['query']
},
readOnlyHint: true,
execute: async ({ query }) => ({
content: [{ type: 'text', text: JSON.stringify(await runLocalDocSearch(query)) }]
})
});
}7. Machine-Readable Errors: RFC 9457 & Agent Recovery
Returning a 40KB styled HTML error page for a rate limit or missing route forces an LLM to burn tokens guessing what went wrong. RFC 9457 application/problem+json turns failures into deterministic instructions.
code and an explicit resolution hint.
RFC 9457 application/problem+json Response Contract
HTTP/1.1 429 Too Many Requests
Content-Type: application/problem+json
Retry-After: 30
{
"type": "https://is-agentic.com/errors/rate-limit-exceeded",
"title": "Rate limit exceeded",
"status": 429,
"detail": "Exceeded 120 requests per 60 seconds from this IP address.",
"code": "rate_limit_exceeded",
"resolution": "Honor the Retry-After header and wait 30 seconds before retrying."
}Stable code |
HTTP | Meaning | Automated Agent Recovery Action |
|---|---|---|---|
invalid_url |
400 |
Target is malformed or private | Validate scheme and pass a public http:// or https:// URL. |
report_not_found |
404 |
No cached scan exists yet | Run npx is-agentic <url> --json or open /scan/<target>. |
rate_limit_exceeded |
429 |
Burst exceeded 120 req / 60s | Read the HTTP Retry-After header and sleep before retrying. |
report_temporarily_unavailable |
503 |
Transient backend hiccup | Retry briefly with backoff; do not trigger duplicate scans. |
8. Five Copy-Paste Recipes for Agent-Ready Websites
Drop these battle-tested templates into documentation sites, SaaS applications, and storefronts to raise your is-agentic score and unlock reliable agent interactions.
| Recipe & Goal | Stack | Implementation Pattern | Verification Step |
|---|---|---|---|
| 1. CI Readiness Gate Block regressions on PR deploy |
GitHub Actions + CLI | npx is-agentic $URL --json parsed with jq or Node threshold check |
Confirm essential.passing == total |
2. Curated /llms.txtZero-noise documentation map |
Static file or build script | Single # H1, > blockquote, grouped - [Title](url): desc links |
Validate 200 OK & valid Markdown links |
| 3. Edge Markdown Switch Token-light page delivery |
Next.js Middleware or CDN | Inspect Accept: text/markdown, rewrite route, set Vary: Accept |
Test curl -H "Accept: text/markdown" |
| 4. Zero-JS Booking Tool Agent-callable HTML forms |
HTML5 + WebMCP attributes | Add toolname, tooldescription, toolparamdescription + e.respondWith |
Verify schema in Chrome WebMCP flag |
| 5. SPA Tool Lifecycle Sync tools with route views |
React / Vue + WebMCP | Call registerTool on mount and unregisterTool(name) on cleanup |
Run clearContext() on sign-out |
Copy-Paste /llms.txt Root Template
# Acme Cloud Platform
> Developer documentation, REST endpoints, and billing guides for Acme Cloud.
## Core Guides
- [Quickstart Guide](https://acme.example/docs/quickstart.md): Deploy your first service in three minutes.
- [Authentication API](https://acme.example/docs/auth.md): Scoped API tokens, OAuth flows, and webhook signing.
## Optional
- [Complete Corpus Export](https://acme.example/llms-full.txt): Single-file documentation bundle for bulk indexing.Copy-Paste CI/CD Readiness Gate (Shell + Node One-Liner)
REPORT=$(npx is-agentic https://staging.acme.example --json)
node -e "const r=JSON.parse(process.argv[1]); if (r.score !== null && r.score < 85) process.exit(1);" "$REPORT"9. Security Boundaries, Rate Limits & Human Confirmation Gates
Because browser WebMCP tools run directly inside an authenticated user’s session with live cookies, high-stakes actions like purchases or account deletions must never execute silently.
is-agentic.com enforces 120 requests per IP per 60 seconds. Cache JSON results locally in CI and obey HTTP Retry-After headers on 429 responses.
Reports are immutable snapshots that refresh only when a visit finds them older than 6 hours. Compare scanned_at after deploys or trigger manual Rescan.
Private MCP servers yield score: null. Keep internal admin dashboards out of /llms.txt and sitemap.xml while allowing public docs.
Human-in-the-Loop Checkout Gate Using client.requestUserInteraction()
navigator.modelContext.registerTool({
name: 'place_order',
description: 'Submit the active shopping cart for final checkout after user confirmation.',
inputSchema: {
type: 'object',
properties: { shippingMethod: { type: 'string', enum: ['standard', 'express'] } },
required: ['shippingMethod']
},
consequentialHint: true,
execute: async ({ shippingMethod }, client) => {
const approved = await client.requestUserInteraction(async () => {
return window.confirm('Confirm checkout using ' + shippingMethod + ' shipping?');
});
if (!approved) {
throw new Error('Checkout cancelled by user.');
}
const res = await fetch('/api/checkout', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ shippingMethod })
});
const order = await res.json();
return { content: [{ type: 'text', text: 'Confirmed Order ID: ' + order.orderId }] };
}
});10. Cross-Tool Synergy: From Source Docs to Verified Browser Actions
When static discovery files, edge Markdown negotiation, automated is-agentic audits, and browser WebMCP tools work in concert, agents spend fewer tokens reading and never break on brittle UI selectors.
is-agentic MCP Loop
Wire https://is-agentic.com/mcp into Claude Code, Cursor, or Antigravity alongside npx skills add vercel-labs/is-agentic. Your agent pulls failing Essential checks via is_agentic_get_report, edits your Next.js middleware or /llms.txt generator, deploys, and verifies the improvement.
First, agents read pricing, return policies, and API guides cleanly through /llms.txt and Accept: text/markdown. Once the user visits the interactive page in Chrome or Edge, WebMCP takes over to invoke search filters, cart updates, and gated checkouts without fragile DOM clicking.
Connecting Your IDE or Agent Host via Streamable HTTP MCP
{
"mcpServers": {
"is-agentic": {
"type": "http",
"url": "https://is-agentic.com/mcp"
}
}
}11. Troubleshooting Clinic & 6-Point Pre-Flight Checklist
Fast, evidence-backed fixes for the seven most common agentic web integration pitfalls, paired with a pre-flight readiness checklist before you ship to production.
| Symptom & Error | Root Cause | 10-Second Production Fix |
|---|---|---|
| Old score after deploying fix | Cached snapshot (< 6 hours old) | Compare scanned_at; click Rescan on report_url for an immediate re-run. |
404 report_not_found |
Read-only REST API never scans | Run npx is-agentic <url> --json or open /scan/<target> first. |
score: null on MCP target |
Endpoint requires authentication | Check score_label; publish public docs or unauthenticated metadata card. |
| CDN shows Markdown to humans | Missing Vary: Accept header |
Set Vary: Accept on every negotiated response and purge edge cache. |
content-no-js Essential fail |
Client-only CSR renders blank shell | Enable SSR or static prerendering (SSG) for core content routes. |
modelContext is undefined |
Missing Chrome flag or HTTPS | Enable #enable-webmcp-testing, serve over HTTPS, and guard with 'modelContext' in navigator. |
429 rate_limit_exceeded |
Exceeded 120 req / 60s IP limit | Read the Retry-After header, back off, and cache reports across CI jobs. |
6-Point Pre-Flight Agent-Readiness Checklist
□ 1. Essential Tier Green: Run npx is-agentic <domain> --json and confirm zero failed or partial checks inside score_breakdown.essential.
□ 2. Curated /llms.txt Live: Confirm GET /llms.txt returns HTTP 200 with a crisp # H1, blockquote summary, and valid Markdown links.
□ 3. Safe Cache Negotiation: Test Accept: text/markdown and Accept: text/html back-to-back to verify Vary: Accept isolates both caches.
□ 4. RFC 9457 Error Contracts: Trigger 400, 404, and 429 paths with an agent header and confirm clean application/problem+json payloads.
□ 5. WebMCP Lifecycle Clean: Verify declarative forms return e.respondWith() and SPA views call unregisterTool() on unmount.
□ 6. Human Checkout Gate Tested: Trigger consequential actions and verify requestUserInteraction() blocks execution until explicit user approval.