Most web scraping tools give your agent one of two bad outputs:
webclaw is an MCP project maintained in the 0xmassi/webclaw repository. The maintainer describes its purpose as “Most web scraping tools give your agent one of two bad outputs”. Read the project documentation for the complete scope.
The documented npm package is @webclaw/mcp. The configuration below comes from the maintainer’s example; adapt its paths and account values to your environment.
Project documentation · Repository · Discovery listing. This profile records the public documentation checked on 2026-10-08.
Check the linked publisher documentation and your connected client's tool list. Names and availability can vary with the server version and account permissions.
| Tool | What it does |
|---|---|
| scrape | Extract one URL as markdown, text, JSON, LLM format, or HTML |
| crawl | Follow same-origin links and extract discovered pages |
| map | Discover URLs without extracting every page |
| batch | Scrape multiple URLs in parallel |
| extract | Convert page content into structured data |
| summarize | Summarize a page |
| diff | Compare page content snapshots |
| brand | Extract colors, fonts, logos, and metadata |
| search | Search the web and scrape results |
| research | Multi-source research workflow |
{
"mcpServers": {
"webclaw": {
"command": "npx",
"args": [
"-y",
"@webclaw/mcp"
]
}
}
}From the project documentation. Replace placeholders and example paths before connecting.
Retrieve public web pages as readable Markdown with the reference Fetch MCP server, including pagination for longer responses.
Use Firecrawl MCP to retrieve website content and extract information for research through Firecrawl’s API-backed web tools.
Yandex search results, images, and SERP data via the Apify Yandex Search Scraper, hosted MCP.
Audit a website for SEO and AI search visibility (ChatGPT, Perplexity, Gemini) and read the fixes.
Audit any site's AI visibility from your assistant: crawler access, rendering, and schema.
Independent-index web search for assistants — fresh results without the Google API maze.