How do I install the llmtest mcp mcp server?

Use a recorded configuration for your client, replace example credential values and paths, and verify the documented runtime. Restart or reconnect the client, then inspect the available tools.

What credentials does it need?

It reads `LLMTEST_API_KEY`, `LLMTEST_BASE_URL` from the environment. Store these in your client's env block or a secrets manager — never commit them to a repository — and grant the narrowest scope that still covers your use case.

SSyncDev

Start a project

Llmtest MCP Server

LLMTest MCP server — benchmark AI models and get cost-saving suggestions in your IDE

Local serverstdioGo

What is the Llmtest MCP Server?

LLMTest MCP server — benchmark AI models and get cost-saving suggestions in your IDE

What you get

MCP server that benchmarks AI models on your actual prompts and finds cheaper, faster alternatives. Works with Claude Code, Cursor, Windsurf, and any MCP-compatible tool.

More MCP options

Compare MCP servers by category, or browse hosted MCP servers if you prefer an endpoint over a local process.

Maintainer setup documentation.

Recorded tools

Check the linked publisher documentation and your connected client's tool list. Names and availability can vary with the server version and account permissions.

Tool	What it does
status	Show proxy status and activity summary
list_flows	List all AI flows with cost and latency stats
get_suggestions	Get pending model-switch recommendations
update_suggestion	Accept or dismiss a suggestion
run_benchmark	Benchmark a flow against challenger models
optimize_prompt	Rewrite a flow's prompt and find a cheaper model that still works
seed_samples	Add test prompts for pre-launch benchmarking
list_samples	Show stored test samples per flow
list_new_models	Show new and trending models
get_account	Check credit balance and usage
get_autopilot_status	Check whether autopilot is on and whether the account is eligible
enable_autopilot	Turn on weekly auto-optimization with safety gates + drift-based auto-revert
disable_autopilot	Turn off autopilot (existing optimizations stay active)
list_active_optimizations	List auto-accepted optimizations still inside their 24h revert window
revert_optimization	Roll an auto-accepted optimization back to the previous prompt

How to install the Llmtest MCP Server

{
  "mcpServers": {
    "llmtest": {
      "command": "npx",
      "args": [
        "llmtest-mcp"
      ],
      "env": {
        "LLMTEST_API_KEY": "YOUR_LLMTEST_API_KEY"
      }
    }
  }
}

From the project documentation. Replace placeholders and example paths before connecting.

Configuration

Variable	Description	Required
LLMTEST_API_KEY	Declared in the maintainer’s client configuration.	Check documentation

Example prompts to try

“Use Llmtest MCP to status.”
“Use Llmtest MCP to list flows.”
“Use Llmtest MCP to get suggestions.”

Frequently asked questions

LLMTest MCP server — benchmark AI models and get cost-saving suggestions in your IDE Confirm the tools offered by your installed version in the connected client. Tool availability and permissions depend on the server and account configuration.