Prompt evals over MCP: run a prompt on your dataset, score each output 1-5 with an LLM judge.
CompletionKit MCP server exists for a simple reason — assistants are far more useful when they can act on CompletionKit directly instead of describing what you should do. Prompt evals over MCP: run a prompt on your dataset, score each output 1-5 with an LLM judge.
Run every prompt against real data. Score each output with an LLM judge against criteria you define. Change anything: the prompt, the model, the temperature, the dataset. Re-run and see exactly what got better and what broke. When the scores tell you something's off, CompletionKit suggests an improved prompt based on the judge's actual feedback on your runs. You inspect the diff, apply it as a new version, and verify the improvement.
Being a remote server, there is no local install. You register the endpoint with your client, authorise it once, and the tools appear.
Once CompletionKit is connected, these are the calls the assistant has available:
Encryption — Provider API keys are encrypted at rest using Active Record encryption. You needYou will need 8 environment variables: OPENAI_API_KEY, ANTHROPIC_API_KEY, OPENROUTER_API_KEY, COMPLETION_KIT_ENCRYPTION_PRIMARY_KEY, COMPLETION_KIT_ENCRYPTION_DETERMINISTIC_KEY, COMPLETION_KIT_API_TOKEN, YOUR_TOKEN, DATABASE_URL. The server will not start without them, which is usually why the tools fail to appear on a first run. Keep credentials in your client's env block or a secrets manager rather than in a file you might commit.
This sits in the AI and media services group, where several servers overlap in what they claim to do but differ sharply once you actually set them up. CompletionKit's toolset — Encryption — is a fair guide to whether it matches your workflow. It is maintained by com.completionkit; worth a glance at recent repository activity before you build anything load-bearing on it.
This entry was verified against CompletionKit's own documentation before publication; SyncDev keeps the directory reviewed rather than auto-generated.
| Tool | What it does |
|---|---|
| Encryption | Provider API keys are encrypted at rest using [Active Record encryption](https://guides.rubyonrails.org/active_record_encryption.html). You need three encryption keys configured before the app will boot in production. |
{
"mcpServers": {
"completion-kit": {
"url": "https://your-app.com/completion_kit/mcp",
"headers": { "Authorization": "Bearer YOUR_TOKEN" }
}
}
}Configuration as documented by the project. Restart the client after saving.
| Variable | Description | Required |
|---|---|---|
| OPENAI_API_KEY | Credential the server authenticates with. | Yes |
| ANTHROPIC_API_KEY | Credential the server authenticates with. | Yes |
| OPENROUTER_API_KEY | Credential the server authenticates with. | Yes |
| COMPLETION_KIT_ENCRYPTION_PRIMARY_KEY | Credential the server authenticates with. | Yes |
| COMPLETION_KIT_ENCRYPTION_DETERMINISTIC_KEY | Credential the server authenticates with. | Yes |
| COMPLETION_KIT_API_TOKEN | Credential the server authenticates with. | Yes |
| YOUR_TOKEN | Credential the server authenticates with. | Yes |
| DATABASE_URL | Endpoint or connection string the server talks to. | Yes |
Build a programmable telecommunications stack for connecting telephony services with the Internet via a cloud-based utility.
Search built for AI, not humans — semantic web search that returns model-ready content, plus code context.
Answers, not links — delegate questions to Perplexity's search-grounded models and get cited responses back.
Give your assistant a voice — text-to-speech, voice cloning and audio tools from the ElevenLabs API.
Give your assistant a real code sandbox — isolated cloud VMs for actually running the code it writes.
The ML hub in your context window — search models, datasets, papers and run Spaces from the official server.