The Okareo MCP server exposes Okareo's evaluation capabilities as MCP tools, allowing AI coding assistants to create and manage scenarios, register models, run evaluations, and execute multi-turn…
Okareo is an MCP project maintained in the okareo-ai/okareo-mcp repository. The maintainer describes its purpose as “The Okareo MCP server exposes Okareo's evaluation capabilities as MCP tools, allowing AI coding assistants to create and manage scenarios, register models, run evaluations, and…”. Read the project documentation for the complete scope.
The maintainer’s configuration uses https://tools.okareo.com/mcp. It declares streamable-http transport. Check the project’s account and authentication instructions when adding the endpoint to your client.
Project documentation · Repository · Discovery listing. This profile records the public documentation checked on 2026-10-08.
Check the linked publisher documentation and your connected client's tool list. Names and availability can vary with the server version and account permissions.
| Tool | What it does |
|---|---|
| save_scenario | Save a named scenario from rows of input/result data (idempotent) |
| list_scenarios | List all scenarios in the project with names, IDs, and row counts |
| get_scenario | Retrieve a scenario's metadata and all data rows by name or ID |
| create_scenario_version | Create a new version of an existing scenario with updated data |
| preview_delete_scenario | Preview what will be deleted before removing a scenario |
| delete_scenario | Permanently delete a scenario and all related test data |
| list_available_llms | Browse available LLMs from the Okareo registry |
| register_generation_model | Register a generation model for testing by selecting an LLM from the registry |
| list_generation_models | List all registered generation models in the project |
| get_generation_model | Read detailed information about a registered generation model |
| update_generation_model | Change the LLM a registered generation model points to |
| delete_generation_model | Remove a registered generation model and all its related test data |
| list_checks | List available quality checks (built-in and custom) for evaluating model outputs |
| run_test | Run a quality test that evaluates a model against a scenario using specified checks |
| list_test_runs | Find past test runs, most recent first. detail_level="summary" (default) or "detailed" for aggregate metrics |
| get_test_run_results | Load one page of a run's results by ID or name. detail_level="summary" (default) / "detailed" (per-check outcomes + explanations) / "full" (transcripts).… |
| get_conversation_transcript | Retrieve the full conversation transcript for a single data point |
| reevaluate_test_run | Re-score a completed test run against a (possibly different) set of checks |
| create_or_update_check | Create or update a quality check by name — model-based, code-based, or audio (upsert) |
| generate_check | Generate a check from a natural-language description, then save it |
| get_check | Retrieve a check's full configuration, including its prompt template or code |
| delete_check | Permanently delete a check by name |
| calibrate_check | Dry-run a draft check against a finished test run — per-row verdicts and the arguments the check received, saving nothing |
| create_or_update_target | Create or update a Target — generation model, custom endpoint, or voice (OpenAI, Deepgram, Twilio) |
| get_target | Retrieve a Target's configuration by name (all types) |
| list_targets | Find simulation targets (voice and custom_endpoint). Bounded to 20 by default; name_contains filters |
| delete_target | Remove a simulation target and all its related test data |
| create_or_update_driver | Define a simulated user persona that will interact with your target |
| get_driver | Retrieve a Driver's full configuration including the persona prompt |
| list_drivers | Find Driver personas. Bounded to 20 by default; name_contains filters |
| list_driver_voices | Discover the voices, voice profiles, and languages available for voice drivers |
| run_simulation | Run a multi-turn conversation evaluation. With based_on_run_id it inherits the original run's full configuration; any argument you pass overrides that one… |
| list_simulations | List past simulation runs with optional filters (target, scenario, limit) |
| ingest_conversations | Submit completed voice conversations to Okareo for monitoring |
| connect_voice_integration | Connect a voice provider so its traffic flows into Okareo monitoring |
| list_voice_integrations | List the voice provider integrations in your project |
| get_voice_integration | Retrieve a voice provider integration by ID, including its status |
| update_voice_integration | Update a voice provider integration's metadata |
| rotate_voice_integration_secret | Rotate a voice provider integration's secrets |
| delete_voice_integration | Delete a voice provider integration by ID |
| get_voice_webhook_url | Get the inbound webhook endpoint for a voice provider |
| list_projects | List every project you can work in — id, name, tags, archive state — with the active one marked |
| select_project | Choose the project to work in for this conversation |
| create_project | Create a new project |
| update_project | Rename a project, or replace its tags |
| archive_project | Hide a project from the project picker. Reversible; nothing is deleted |
| unarchive_project | Restore an archived project to the picker |
| clone_project | Copy an existing project's scenarios into a new project (dry-run first) |
| query_analytics | Query Okareo's product analytics to understand evaluation trends |
| list_dashboards | List the analytics dashboards in your project |
| get_dashboard | Retrieve a dashboard's full configuration by name |
| save_dashboard | Create or update an analytics dashboard by name (upsert) |
| reorder_dashboards | Set the display order of dashboards |
| delete_dashboard | Delete a dashboard by name |
| list_tenants | List every Okareo organization you have access to in this MCP session |
| switch_tenant | Doesn't change the active organization; returns how to reconnect and pick another one at sign-in |
| get_docs | Query the Okareo documentation system for conceptual or user-legible explanations |
| get_templates | Retrieve prompt templates for common Okareo patterns (works offline) |
| get_reps_baseline | Serve REPS agent-evaluation baseline material (scenario banks, drivers, checks, eval configs) from the latest tagged okareo-tools release |
{
"mcpServers": {
"okareo": {
"type": "http",
"url": "https://tools.okareo.com/mcp"
}
}
}From the project documentation. Replace placeholders and example paths before connecting.
The original Chromium automation reference server — simple, screenshot-driven browser control.
Build a programmable telecommunications stack for connecting telephony services with the Internet via a cloud-based utility.
Automates browsers and APIs, enabling Large Language Models (LLMs) to interact with web pages for tasks like screenshot capture and JavaScript
Workers, KV, R2 and D1 by conversation — Cloudflare's official remote servers for building and observability.
Search built for AI, not humans — semantic web search that returns model-ready content, plus code context.
Answers, not links — delegate questions to Perplexity's search-grounded models and get cited responses back.