Web scraper for agents: blocked, empty and wrong pages reported as such, with an Evidence Record.
Octocrawl is an MCP integration. Web scraper for agents: blocked, empty and wrong pages reported as such, with an Evidence Record.
The registry lists a streamable-http connection at https://mcp.octocrawl.dev/mcp.
The published npm package is @octocrawl/mcp, version 0.4.2. Its declared MCP transport is stdio.
The package declares W2L_API_URL: The Octocrawl API the server calls: npx octocrawl serve on this computer (default http://127.0.0.1:8787), or https://api.octocrawl.dev; W2L_API_TOKEN: A bearer token for a hosted or self-hosted API; none for a local one.
Check the maintainer’s instructions for account access, authentication and the supported client configuration. Once connected, inspect the tools your client receives and try a small task that matches the server’s stated purpose. The registry description does not include a verified, complete tool list.
View the publisher’s registry entry · Project repository · Project website. This profile uses version 0.4.2 from the registry. Registration identifies the publisher namespace; it does not establish that this is a first-party integration for every service it connects.
Browse related developer tools MCP servers for tools that cover the same kind of work.
| Variable | Description | Required |
|---|---|---|
| W2L_API_URL | The Octocrawl API the server calls: npx octocrawl serve on this computer (default http://127.0.0.1:8787), or https://api.octocrawl.dev | Check documentation |
| W2L_API_TOKEN | A bearer token for a hosted or self-hosted API; none for a local one | Check documentation |
Read, search and edit files inside allowed directories with the Model Context Protocol reference Filesystem server.
Retrieve version-aware library documentation and code examples with Context7 MCP, using resolve-library-id and query-docs.
Use Microsoft’s Playwright MCP server to navigate pages, inspect accessibility snapshots, fill forms and capture browser screenshots.
Connect GitHub MCP to repositories, issues and pull requests with GitHub’s hosted service or a locally run server.
Use the Supabase MCP server to inspect database schemas, query project data and retrieve docs through a project-scoped connection.
Search Linear issues and projects, inspect work context and manage updates through Linear’s hosted MCP server with OAuth.