Crawl4ai MCP Server

MCP (Model Context Protocol) server for Crawl4AI - Universal web crawling and data extraction. Supports STDIO, SSE, and HTTP transports.

Local serverstdio

What is the Crawl4ai MCP server?

Crawl4ai MCP server exists for a simple reason — assistants are far more useful when they can act on Crawl4ai directly instead of describing what you should do. MCP (Model Context Protocol) server for Crawl4AI - Universal web crawling and data extraction. Supports STDIO, SSE, and HTTP transports.

What you get

Integrate powerful web scraping capabilities into Claude, ChatGPT, and any MCP-compatible AI assistant.

Setting it up

The server ships on npm as crawl4ai-mcp, so your MCP client can launch it on demand — there is no separate build step. Add the server block to your client's configuration, restart it, and the tools register themselves.

Configuration and credentials

You will need 2 environment variables: CRAWL4AI_ENDPOINT, CRAWL4AI_BEARER_TOKEN. The server will not start without them, which is usually why the tools fail to appear on a first run. Keep credentials in your client's env block or a secrets manager rather than in a file you might commit.

Choosing this one

This sits in the developer tooling group, where several servers overlap in what they claim to do but differ sharply once you actually set them up. It is maintained by stgmt; worth a glance at recent repository activity before you build anything load-bearing on it.

This entry was verified against Crawl4ai's own documentation before publication; SyncDev keeps the directory reviewed rather than auto-generated.

Before you rely on it

  • It runs with your machine's permissions. That is convenient and also the reason to think about what you point it at before you approve a tool call.
  • Missing credentials fail quietly in some clients — if no tools show up, check the environment block first.
  • MCP clients confirm each tool call by default. Leave that on until you have watched what the crawl4ai mcp server does with a few real requests.

How to install the Crawl4ai MCP server

{
  "mcpServers": {
    "crawl4ai-1": {
      "command": "npx",
      "args": ["-y", "crawl4ai-mcp"],
      "env": {
        "CRAWL4AI_ENDPOINT": "your-value",
        "CRAWL4AI_BEARER_TOKEN": "your-value"
      }
    }
  }
}

Add to claude_desktop_config.json, then restart Claude Desktop.

Configuration

VariableDescriptionRequired
CRAWL4AI_ENDPOINTConfiguration value read at startup.Optional
CRAWL4AI_BEARER_TOKENCredential the server authenticates with.Yes

Frequently asked questions

It connects Crawl4ai to MCP-compatible AI assistants such as Claude and Cursor. Instead of copying data back and forth by hand, the assistant works with Crawl4ai directly.