SSyncDev

Website Crawler — Markdown Pages, Sitemaps & RAG Provenance MCP Server

Crawl sites to Markdown with sitemaps, globs and depth limits, obeying robots and crawl-delay

Remote serverstreamable-http

What is the Website Crawler — Markdown Pages, Sitemaps & RAG Provenance MCP Server?

Website Crawler — Markdown Pages, Sitemaps & RAG Provenance is an MCP integration. Crawl sites to Markdown with sitemaps, globs and depth limits, obeying robots and crawl-delay.

Hosted connection

The registry lists a streamable-http connection at https://mcp.apify.com/?tools=hermeskd/website-to-markdown. The endpoint declares these request headers: Authorization.

Before connecting

Check the maintainer’s instructions for account access, authentication and the supported client configuration. Once connected, inspect the tools your client receives and try a small task that matches the server’s stated purpose. The registry description does not include a verified, complete tool list.

Project and source

View the publisher’s registry entry · Project website. This profile uses version 1.0.4 from the registry. Registration identifies the publisher namespace; it does not establish that this is a first-party integration for every service it connects.

Browse related search & web scraping MCP servers for tools that cover the same kind of work.