Crawl4ai Skill MCP Server

Scrape JavaScript-heavy sites and extract structured data via reusable CSS schemas. A portable agent skill that wraps the

Local serverstdioPython

What is the Crawl4ai Skill MCP server?

Scrape JavaScript-heavy sites and extract structured data via reusable CSS schemas. A portable agent skill that wraps the Crawl4AI CLI and Python SDK, written in the Anthropic SKILL.md format and consumable by any. Exposed over MCP by the crawl4ai skill mcp server, that capability becomes something an assistant can invoke while it works, not something you go and do afterwards.

What it actually does

  • JS-aware crawling — full headless-browser rendering with wait_until=networkidle defaults
  • Schema-based extraction — derive a CSS selector schema once via LLM, apply it forever with no further LLM cost
  • LLM extraction — per-request structured extraction when a schema is not worth deriving
  • Content filtering — BM25 relevance filter and quality-based pruning, plain markdown or markdown-fit output
  • Concurrent batch crawling — multi-URL processing with per-job concurrency caps
  • Session management — persistent sessions for authenticated, multi-step flows

Its toolset

Everything the assistant can do here goes through one of these:

  • Schema — based extraction**: derive a CSS selector schema once via LLM, apply it forever with no further LLM cost
  • Path — Contents
  • SKILL.md — Entry point: trigger conditions, defaults, routing to specialized pipelines
  • VERSION — Pinned Crawl4AI library version the skill is verified against
  • Script — Purpose

Adding it to your client

crawl4ai on PyPI is all you need. Most clients run it directly, so configuration is a few lines and a restart.

Configuration

The skill calls into the Crawl4AI Python library, which must be installed in the runtime your agent uses: crawl4ai-doctor validates the install and confirms a headless browser is available.

When to reach for it

Plenty of AI and media services servers cover similar ground. The differences that matter in practice are scope of access and how much setup stands between you and a working tool call. Crawl4ai Skill's toolset — Schema, Path, SKILL.md and 2 more — is a fair guide to whether it matches your workflow. It is maintained by brettdavies; worth a glance at recent repository activity before you build anything load-bearing on it.

SyncDev reviews every entry in this directory against the project's own documentation before publishing, and revisits them as servers change.

Caveats

  • It runs with your machine's permissions. That is convenient and also the reason to think about what you point it at before you approve a tool call.
  • MCP clients confirm each tool call by default. Leave that on until you have watched what the crawl4ai skill mcp server does with a few real requests.

Available tools

ToolWhat it does
Schemabased extraction**: derive a CSS selector schema once via LLM, apply it forever with no further LLM cost
PathContents
SKILL.mdEntry point: trigger conditions, defaults, routing to specialized pipelines
VERSIONPinned Crawl4AI library version the skill is verified against
ScriptPurpose

How to install the Crawl4ai Skill MCP server

{
  "mcpServers": {
    "crawl4ai-skill": {
      "command": "uvx",
      "args": ["crawl4ai"]
    }
  }
}

Add to claude_desktop_config.json, then restart Claude Desktop.

Configuration

The skill calls into the Crawl4AI Python library, which must be installed in the runtime your agent uses: crawl4ai-doctor validates the install and confirms a headless browser is available.

Example prompts to try

  • Use Crawl4ai Skill to Schema.
  • Use Crawl4ai Skill to Path.
  • Use Crawl4ai Skill to SKILL.md.

Frequently asked questions

It connects Crawl4ai Skill to MCP-compatible AI assistants such as Claude and Cursor, exposing 5 tools (Schema, Path, SKILL.md, and more) that the assistant can call on your behalf. Instead of copying data back and forth by hand, the assistant works with Crawl4ai Skill directly.