Run local prompt regression checks from prompt-response logs in AI coding assistants.
Run local prompt regression checks from prompt-response logs in AI coding assistants. Exposed over MCP by the redline mcp server, that capability becomes something an assistant can invoke while it works, not something you go and do afterwards.
redline is an open-source, local-first eval tool for AI teams. It uses logs you already have: prompts, outputs, support tickets, traces, model responses, and production JSONL exports.
Everything the assistant can do here goes through one of these:
redline — command not found: run python -m pip install redline-ai, thenRedlineMiddleware — capture bounded JSON FastAPI or ASGI request/response pairs locally, with optional skip diagnosticsredline-mcp — let AI coding assistants run checks inside Claude, Codex,uses — gowtham0992/redline@v0.3.0self — contained HTML for side-by-side inspection, including feature and prompt eval tablesProof — Command or dataDashboard — HTML reportSignal — Example regressionKey — Purposesuite — Suite baseline path, default redline-suite.jsonmax_cases — Maximum representative cases selected for a suitereplay — Command used by eval; prompts go to stdin by default. {prompt} is for small legacy argv runners; {prompt_file} passes a temporary rendered-promptBeing a remote server, there is no local install. You register the endpoint with your client, authorise it once, and the tools appear.
This sits in the monitoring and observability group, where several servers overlap in what they claim to do but differ sharply once you actually set them up. Redline's toolset — redline, RedlineMiddleware, redline-mcp and 11 more — is a fair guide to whether it matches your workflow. It is maintained by gowtham0992; worth a glance at recent repository activity before you build anything load-bearing on it.
We check each listing at SyncDev against the project's documentation before it goes live — if something here drifts out of date, it is a bug worth reporting.
| Tool | What it does |
|---|---|
| redline | command not found: run python -m pip install redline-ai, then |
| RedlineMiddleware | capture bounded JSON FastAPI or ASGI request/response pairs locally, with optional skip diagnostics. |
| redline-mcp | let AI coding assistants run checks inside Claude, Codex, |
| uses | gowtham0992/redline@v0.3.0 |
| self | contained HTML for side-by-side inspection, including feature and prompt eval tables |
| Proof | Command or data |
| Dashboard | HTML report |
| Signal | Example regression |
| Key | Purpose |
| suite | Suite baseline path, default redline-suite.json. |
| max_cases | Maximum representative cases selected for a suite. |
| replay | Command used by eval; prompts go to stdin by default. {prompt} is for small legacy argv runners; {prompt_file} passes a temporary rendered-prompt file path. |
| workers | Number of replay cases to run concurrently. |
| owners | Optional pattern-to-owner rules so regressions show the responsible team. |
{
"mcpServers": {
"redline": {
"command": "uvx",
"args": ["redline-ai"]
}
}
}Add to claude_desktop_config.json, then restart Claude Desktop.
Give your coding agent the full DevTools toolbox: traces, network, console, heap snapshots and Lighthouse.
Dashboards, Prometheus and Loki queries, incidents and alerts — observability by conversation.
Errors with full context — stack traces, issue triage and AI-powered root-cause analysis from Sentry's server.
Enables enhanced web research capabilities for large language models through intelligent search queuing and advanced content extraction.
Automates browser interactions and enables Large Language Models (LLMs) to interact with web pages through Playwright and Chrome DevTools Protocol
Guides tool usage by providing recommendations for MCP tools at each problem-solving stage.