How do I install Crawl Readiness?

Copy the stdio config for Cursor, Claude Desktop, Claude Code, or VS Code on this page. It uses the publisher's install, not a PlugMCP proxy.

Does Crawl Readiness need an API key?

The publisher lists these environment variables: CRAWL_READINESS_API_KEY, CRAWL_READINESS_API_BASE.

What transport does Crawl Readiness use?

Crawl Readiness runs locally over stdio.

When was Crawl Readiness last checked?

Last checked 2026-11-09. Check what AI crawlers may read and what a page permits, generate llms.txt and schema.

Crawl Readiness MCP server

Power your agent

Add this server

The publisher's own command. Not a PlugMCP endpoint.

Add to Cursor Add to VS Code Run on PlugMCP

Host a copy with our keys and logs.

Local command

npx -y [email protected]

Credentials from the publisher

CRAWL_READINESS_API_KEY — Free API key from https://www.crawlreadiness.com/dashboard. The four audit tools and check_permission work without it; the three generators need it, and get_monitor_trend needs one on a plan with Answer Analyses.
CRAWL_READINESS_API_BASE — Override the API base URL. Only useful for testing against a preview deployment; defaults to https://www.crawlreadiness.com.

Claude Code has no install link for a local command. Copy the command below and run it in a terminal.

shClaude Code

1CRAWL_READINESS_API_KEY=<secret> CRAWL_READINESS_API_BASE=<value> claude mcp add io.github.grubbomatic-crawl-readiness -- npx -y crawl-readiness-mcp@0.2.9

Run in a terminal.

jsonClaude Desktop

1{2  "mcpServers": {3    "io.github.grubbomatic-crawl-readiness": {4      "command": "npx",5      "args": [6        "-y",7        "[email protected]"8      ],9      "env": {10        "CRAWL_READINESS_API_KEY": "<secret>",11        "CRAWL_READINESS_API_BASE": "<value>"12      }13    }14  }15}

claude_desktop_config.json

Local · npmPublisher installStep-by-step: add this in Claude

Remote URL or a command on your machine? How the two installs differ

Run on PlugMCP

Host a copy with our keys and logs.

A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.

Launch

Server facts

Local

Transport: stdio
Version

Tools

No tool list yet. Remote servers get one after the first health check.

FAQ

Questions

Crawl Readiness MCP Server

An MCP server that checks a website's AI readiness: can ChatGPT, Claude, Perplexity, Google AI and 50+ other AI crawlers access it? Then it generates the fixes (llms.txt, an AI-crawler robots.txt, JSON-LD structured data) and reads your LLM Monitor results, inside Claude Desktop, Cursor, Cline, Windsurf and other MCP clients.

Ask Claude "is my site AI-readable?" and it'll actually check — running a live audit against 50+ AI crawlers (ChatGPT, Claude, Perplexity, Google AI, and more). Then ask it to fix what's broken, and it can generate a proper llms.txt, robots.txt, or JSON-LD schema and write it straight into your project.

No more copy-paste between your editor and yet another SEO tool.

What it does

Eight tools your AI assistant can call:

Audit (no signup):

check_ai_readiness — Score a site 0–100 against 50+ AI crawlers, with a prioritized fix list; extras that don't affect the score are marked as such
validate_schema — Audit JSON-LD structured data on any URL (20+ types, per-type rules)
validate_robots — Line-by-line robots.txt audit
check_content_parity — Compare what humans see vs what AI crawlers see

Generate (requires free API key):

generate_llms_txt — Auto-crawl a site and produce a properly formatted llms.txt
generate_robots_txt — AI-crawler-aware robots.txt with preset policies
generate_schema — Organization / WebSite / Article JSON-LD ready to paste

Monitor (read-only, requires free API key):

get_monitor_trend — See whether ChatGPT, Claude, Perplexity & Google AI mention your brand vs competitors, and how that's trending week over week (reads your LLM Monitor projects; doesn't trigger runs)

Install

Claude Desktop, one click

Download crawl-readiness-mcp-<version>.mcpb from the latest release and double-click it (or drag it into the Claude Desktop window). Claude Desktop shows what the extension does and asks for the optional API key. The bundle carries the server and its dependencies, so nothing else is installed.

Claude Code, as a plugin

This repository is also a Claude plugin: the .mcp.json at its root starts the server (pinned to this version), and the ai-readiness skill tells Claude when to use which tool and how to read the results. Add it from the Claude directory once it is listed there. To try it before then, clone the repository and start Claude Code with it loaded:

claude --plugin-dir ./crawl-readiness-mcp

Claude Code asks for the optional API key when the plugin is enabled and keeps it in secure storage; the plugin's own files never contain it.

For Claude Desktop, by hand

Open your config file:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json

Add this to the mcpServers block:

{
  "mcpServers": {
    "crawl-readiness": {
      "command": "npx",
      "args": ["-y", "crawl-readiness-mcp"]
    }
  }
}

Restart Claude Desktop. You'll see "crawl-readiness" listed in the tools panel.

For Cursor

Open Cursor Settings → Features → Model Context Protocol → Add new global MCP server.

{
  "crawl-readiness": {
    "command": "npx",
    "args": ["-y", "crawl-readiness-mcp"]
  }
}

For Cline / Windsurf / Continue

Same shape — add crawl-readiness to your MCP servers list with npx -y crawl-readiness-mcp as the command.

Unlock the generator tools

The four audit tools work out of the box, no signup. The three generator tools need an API key — free at crawlreadiness.com/dashboard (50 checks/month on the free tier, no credit card).

Once you have a key, add it via env in the same config:

{
  "mcpServers": {
    "crawl-readiness": {
      "command": "npx",
      "args": ["-y", "crawl-readiness-mcp"],
      "env": {
        "CRAWL_READINESS_API_KEY": "cr_live_xxxxxxxxxxxx"
      }
    }
  }
}

Restart the client. All eight tools are now callable.

Example prompts

Once installed, try any of these in your AI client:

"Is my portfolio site AI-readable? URL: yourdomain.com"
"Check if ChatGPT can see acme-inc.com"
"Validate the JSON-LD on stripe.com — is anything missing?"
"Compare what humans see vs what AI crawlers see on producthunt.com"
"Generate an llms.txt for my site and save it to /public/llms.txt"
"My site scored 68/100 — walk me through the fixes"

Your AI can chain tools automatically — audit, identify issues, fix them, and write files to your project without you copy-pasting anything.

Configuration

Env var	Purpose	Default
`CRAWL_READINESS_API_KEY`	Free API key for generator tools	none
`CRAWL_READINESS_API_BASE`	Override the API base URL (for testing)	`https://www.crawlreadiness.com`

What tools receive from the server

Each tool returns the full JSON response from the Crawl Readiness API — the same data the web dashboard renders. Your AI has access to:

check_ai_readiness: score, robots.txt status per crawler, structured-data flags, homepage signals, prioritized fix list (fixes marked extra: true are optional and not scored)
validate_schema: per-block issues (errors, warnings, info), suggested missing types
validate_robots: line-by-line issues, AI-bot coverage summary
check_content_parity: verdict + per-crawler HTTP status, word overlap %, warnings
generate_llms_txt: the file content + companion robots.txt snippet
generate_robots_txt: merged robots.txt with the AI policy applied
generate_schema: JSON-LD scripts for each detected schema type
get_monitor_trend: per-brand mention rate, share of voice, per-provider breakdown, competitor comparison, trend over time, and short example answers

What this sends, and where

Every tool makes HTTPS requests to https://www.crawlreadiness.com and nowhere else. What leaves your machine is the website address (and for the generators, the inputs) you asked about, the API key if you configured one, and a crawl-readiness-mcp/<version> user agent. The server stores nothing locally, runs no other programs, and reads no files; writing a generated file into your project is something your AI client does, with your say-so. The service's own data handling is described at crawlreadiness.com/privacy.

The plugin's .mcp.json starts the published npm package pinned to the same version as the repository (npx -y crawl-readiness-mcp@<version>), so what runs is the code in this repository at that tag.

Support

Docs: crawlreadiness.com/mcp
Issues: github.com/Grubbomatic/crawl-readiness-mcp/issues
Email: [email protected]

About

Crawl Readiness is a suite of AI-SEO tools that check whether AI systems can access your website and help you fix what's blocked. Every generator tool in this MCP server is also available on the web at crawlreadiness.com/tools. The MCP server just makes the same tools callable from inside your AI assistant, so the "audit → fix → apply" loop happens in one conversation without leaving your editor.

Releasing (maintainers)

npm run release validates server.json and prints the publish steps. The last step uses the mcp-publisher CLI, which is not part of this repository: download it from the MCP Registry releases and keep it on your PATH or in the repo root (it is git-ignored there).

Crawl Readiness

Add this server

Host a copy with our keys and logs.

Server facts

Tools

Questions

How do I install Crawl Readiness?

Does Crawl Readiness need an API key?

What transport does Crawl Readiness use?

When was Crawl Readiness last checked?

Documentation

Crawl Readiness MCP Server

What it does

Install

Claude Desktop, one click

Claude Code, as a plugin

For Claude Desktop, by hand

For Cursor

For Cline / Windsurf / Continue

Unlock the generator tools

Example prompts

Configuration

What tools receive from the server

What this sends, and where

Support

About

Releasing (maintainers)

License