Crawlbase MCP Server connects AI agents and LLMs with real-time web data. It powers Claude, Cursor, and Windsurf integrations with battle-tested web
Most search and retrieval work still happens through a UI a human drives. Crawlbase Web MCP server moves it into the conversation instead. Crawlbase MCP Server connects AI agents and LLMs with real-time web data. It powers Claude, Cursor, and Windsurf integrations with battle-tested web scraping, JavaScript rendering, and anti-bot protection enabling structured, live data.
Crawlbase MCP is a Model Context Protocol (MCP) server that bridges AI agents and the live web. Instead of relying on outdated training data, your LLMs can now fetch fresh, structured, real-time content — powered by Crawlbase’s proven crawling infrastructure trusted by 70,000+ developers worldwide.
Setup follows the usual MCP pattern — install or clone the server, register it in your client's configuration file, restart the client. The configuration blocks on this page cover the common clients.
Configuration is passed through the environment: CRAWLBASE_TOKEN, CRAWLBASE_JS_TOKEN. Treat anything key-shaped as a real credential — scope it to the minimum the server needs, and rotate it if it ever lands in a shared config.
This sits in the search and retrieval group, where several servers overlap in what they claim to do but differ sharply once you actually set them up. It is maintained by crawlbase; worth a glance at recent repository activity before you build anything load-bearing on it.
We check each listing at SyncDev against the project's documentation before it goes live — if something here drifts out of date, it is a bug worth reporting.
{
"mcpServers": {
"crawlbase": {
"type": "stdio",
"command": "npx",
"args": ["@crawlbase/mcp@latest"],
"env": {
"CRAWLBASE_TOKEN": "your_token_here",
"CRAWLBASE_JS_TOKEN": "your_js_token_here"
}
}
}
}Configuration as documented by the project. Restart the client after saving.
| Variable | Description | Required |
|---|---|---|
| CRAWLBASE_TOKEN | Credential the server authenticates with. | Yes |
| CRAWLBASE_JS_TOKEN | Credential the server authenticates with. | Yes |
The simplest web tool that matters — fetch any URL and get model-ready markdown back.
Industrial-strength web extraction — render, scrape, crawl and search entire sites into clean markdown.
Puppeteer-powered browser control that drives pages from the accessibility tree instead of pixels.
Search built for AI, not humans — semantic web search that returns model-ready content, plus code context.
Answers, not links — delegate questions to Perplexity's search-grounded models and get cited responses back.
Put 6,000+ pre-built scrapers at your assistant's fingertips through one MCP endpoint.