Download website to a local directory (including all css, images, js, etc.)
Download website to a local directory (including all css, images, js, etc.)
Compare MCP servers by category, or browse hosted MCP servers if you prefer an endpoint over a local process.
Check the linked publisher documentation and your connected client's tool list. Names and availability can vary with the server version and account permissions.
| Tool | What it does |
|---|---|
| urls | Array of objects which contain urls to download and filenames for them. **_Required_**. javascript scrape({ urls: [ 'http://nodejs.org/', // Will be saved with default filename 'index.html' {url: 'http://nodejs.org/about |
| directory | String, absolute path to directory where downloaded files will be saved. Directory should not exist. It will be created by scraper. **_Required_**. How to download website to existing directory and why it's not supported |
| sources | Array of objects to download, specifies selectors and attribute values to select files for downloading. By default scraper tries to download all possible resources. Scraper uses cheerio to select html elements so selecto |
| recursive | Boolean, if true scraper will follow hyperlinks in html files. Don't forget to set maxRecursiveDepth to avoid infinite downloading. Defaults to false. |
| maxRecursiveDepth | Positive number, maximum allowed depth for hyperlinks. Other dependencies will be saved regardless of their depth. Defaults to null - no maximum recursive depth set. |
| maxDepth | Positive number, maximum allowed depth for all dependencies. Defaults to null - no maximum depth set. In most of cases you need [maxRecursiveDepth](#maxRecursiveDepth) instead of this option. |
| request | Object, custom options for http module [got](https://github.com/sindresorhus/got#options) which is used inside website-scraper. Allows to set retries, cookies, userAgent, encoding, etc. javascript // use same request opt |
| subdirectories | Array of objects, specifies subdirectories for file extensions. If null all files will be saved to directory. javascript /* Separate files into directories: - img for .jpg, .png, .svg (full path /path/to/save/img) - js f |
| defaultFilename | |
| prettifyUrls | Boolean, whether urls should be 'prettified', by having the defaultFilename removed. Defaults to false. |
| ignoreErrors | Boolean, if true scraper will continue downloading resources after error occurred, if false - scraper will finish process and return error. Defaults to false. |
| urlFilter | Function which is called for each url to check whether it should be scraped. Defaults to null - no url filter will be applied. javascript // Links to other websites are filtered out by the urlFilter scrape({ urls: ['http |
| filenameGenerator | String (name of the bundled filenameGenerator). Filename generator determines path in file system where the resource will be saved. |
| requestConcurrency | Number, maximum amount of concurrent requests. Defaults to Infinity. |
{
"mcpServers": {
"website-scraper": {
"command": "npx",
"args": ["-y", "website-scraper"],
"env": {
"DEBUG": "your-value"
}
}
}
}Add to claude_desktop_config.json, then restart Claude Desktop.
| Variable | Description | Required |
|---|---|---|
| DEBUG | Configuration value read at startup. | Optional |
Retrieve public web pages as readable Markdown with the reference Fetch MCP server, including pagination for longer responses.
Use Firecrawl MCP to retrieve website content and extract information for research through Firecrawl’s API-backed web tools.
The original Chromium automation reference server — simple, screenshot-driven browser control.
Puppeteer-powered browser control that drives pages from the accessibility tree instead of pixels.
Automates browsers and APIs, enabling Large Language Models (LLMs) to interact with web pages for tasks like screenshot capture and JavaScript
Search built for AI, not humans — semantic web search that returns model-ready content, plus code context.