Can AI crawlers read this site? GPTBot/ClaudeBot verdicts, robots vs edge blocking, 47k-domain index
Ai-crawler-readiness is an MCP integration. Can AI crawlers read this site? GPTBot/ClaudeBot verdicts, robots vs edge blocking, 47k-domain index.
The registry lists a streamable-http connection at https://shop.lumnika.com/ai-readiness/mcp.
Check the maintainer’s instructions for account access, authentication and the supported client configuration. Once connected, inspect the tools your client receives and try a small task that matches the server’s stated purpose. The registry description does not include a verified, complete tool list.
View the publisher’s registry entry · Project website. This profile uses version 1.0.0 from the registry. Registration identifies the publisher namespace; it does not establish that this is a first-party integration for every service it connects.
Browse related search & web scraping MCP servers for tools that cover the same kind of work.
Retrieve public web pages as readable Markdown with the reference Fetch MCP server, including pagination for longer responses.
Use Firecrawl MCP to retrieve website content and extract information for research through Firecrawl’s API-backed web tools.
Claude can perform Web Search | Exa with MCP (Model Context Protocol)
Free open-source web search and page reading for LLMs, AI agents, and RAG.
Read-only Google Search Console MCP server for AI assistants.
Find open-source libraries and fetch contextual code snippets by version to accelerate development…