<em>Web Crawling and RAG Capabilities for AI Agents and AI Coding Assistants</em>
Web Crawling and RAG Capabilities for AI Agents and AI Coding Assistants. That is what the crawl4ai rag mcp mcp server brings to an AI assistant: the same capability, reachable through the Model Context Protocol rather than a separate app or dashboard.
This MCP server provides tools that enable AI agents to crawl websites, store content in a vector database (Supabase), and perform RAG over the crawled content. It follows the best practices for building MCP servers based on the Mem0 MCP server template I provided on my channel previously.
mcp/crawl4ai-rag on a container image is all you need. Most clients run it directly, so configuration is a few lines and a restart.
The server publishes 1 tool. What each one is for:
Workflow — 1. Repository Parsing: Use parse_github_repository tool to clone and analyze open-source repositories 2. Code Validation: UseConfiguration is passed through the environment: TRANSPORT, OPENAI_API_KEY, SUPABASE_URL, SUPABASE_SERVICE_KEY, USE_KNOWLEDGE_GRAPH, NEO4J_URI, NEO4J_USER, NEO4J_PASSWORD. Treat anything key-shaped as a real credential — scope it to the minimum the server needs, and rotate it if it ever lands in a shared config.
This sits in the AI and media services group, where several servers overlap in what they claim to do but differ sharply once you actually set them up. Crawl4ai Rag MCP's toolset — Workflow — is a fair guide to whether it matches your workflow. It is maintained by jrmatherly; worth a glance at recent repository activity before you build anything load-bearing on it.
We check each listing at SyncDev against the project's documentation before it goes live — if something here drifts out of date, it is a bug worth reporting.
| Tool | What it does |
|---|---|
| Workflow | 1. **Repository Parsing**: Use parse_github_repository tool to clone and analyze open-source repositories 2. **Code Validation**: Use check_ai_script_hallucinations tool to validate AI-generated Python scripts 3. **Knowl |
{
"mcpServers": {
"crawl4ai-rag": {
"transport": "sse",
"url": "http://localhost:8051/sse"
}
}
}Configuration as documented by the project. Restart the client after saving.
| Variable | Description | Required |
|---|---|---|
| TRANSPORT | Configuration value read at startup. | Optional |
| OPENAI_API_KEY | Credential the server authenticates with. | Yes |
| SUPABASE_URL | Endpoint or connection string the server talks to. | Yes |
| SUPABASE_SERVICE_KEY | Credential the server authenticates with. | Yes |
| USE_KNOWLEDGE_GRAPH | Configuration value read at startup. | Optional |
| NEO4J_URI | Configuration value read at startup. | Optional |
| NEO4J_USER | Configuration value read at startup. | Optional |
| NEO4J_PASSWORD | Configuration value read at startup. | Optional |
Read-only SQL access to Postgres — let your assistant inspect schemas and answer questions from real data.
Manage your whole Supabase project in conversation — database, auth, storage, Edge Functions and branches.
Query, modify and analyse local SQLite databases in conversation — the fastest way to chat with a data file.
Metabase ships its own MCP endpoint — search your BI content, build and run queries, and save questions and dashboards without leaving the chat.
Official MongoDB server covering data, schemas and Atlas management — from find queries to spinning up clusters.
Serverless Postgres with database branching — point your assistant at Neon and let it work on disposable copies.