ai.zerowidth/caliper

up checked 2026-11-11T00:37:57.827Z

Unclaimed · Unverified

Agent searches it appeared in, last 30 days: 0. Run this server? Claim the listing: verified label, uptime alerts and analytics.

Publisher-provided title

ZeroWidth Caliper

Publisher-provided description

Build datasets and rubrics, then run evals that score your AI in Caliper.

Summary by lookupd

Lets an agent manage AI evaluation datasets, rubrics and scheduled eval runs, and inspect traces, source feeds and flow performance.

Install

Claude Code, remote (streamable-http)

claude mcp add --transport http caliper https://api.zerowidth.ai/mcp/x/caliper

Codex, remote (streamable-http)

codex mcp add caliper --url https://api.zerowidth.ai/mcp/x/caliper

Cursor, remote (streamable-http) → .cursor/mcp.json

{
  "mcpServers": {
    "caliper": {
      "url": "https://api.zerowidth.ai/mcp/x/caliper"
    }
  }
}

Other clients: docs. Or tell your agent: Set up lookupd for me: read https://lookupd.com/agent.md and follow it.

Tools (showing 30 of 45)

Publisher-provided text

• search_docs Search ZeroWidth product documentation. Returns matching pages with title, slug, public URL, and a query-relevant snippet. Use this when the user asks about a Z
• get_doc Fetch the full Markdown body of a specific docs page by its slug. Use this after `search_docs` when the user needs the complete content of a page. No authentica
• list_docs Enumerate all available docs pages, optionally filtered by product (e.g. 'compass', 'legal', 'overview'). Use this to discover what slugs exist before calling `
• caliper_datasets_get One dataset with a page of its items — read this BEFORE editing items (caliper_datasets_update_item needs the item id) and before extending coverage, so you don
• caliper_datasets_update Changes a dataset's title, description, or visibility. Pass only what changes; items are untouched (use caliper_datasets_add_items / caliper_datasets_update_ite
• caliper_datasets_delete Deletes a dataset. Evals bound to it stop being runnable (their binding shows datasetOk: false), and reviews/specs over its items lose their source — check cali
• caliper_datasets_update_item Rewrites one item's content, keeping its id (ratings and eval results keyed to it stay attached). The item's KIND can't change — a Q&A item stays Q&A, a chat it
• caliper_datasets_remove_item Drops one item. Ratings and run results keyed to it are orphaned (kept in history, gone from the dataset). A dataset must keep at least one item. May return `ne
• caliper_datasets_generate Creates a NEW dataset of model-written items — Q&A by default, or scripted sequences / simulated people via `shape` — pass flowId and the generator reads the fl
• caliper_datasets_generate_items Appends model-written items to an EXISTING dataset, in the style of what's already there (existing items are the few-shot examples; the output shape matches the
• caliper_rubrics_list Lists the rubrics in the active workspace (id, name, description, criterion count). Check here BEFORE caliper_rubrics_create — reuse an existing rubric's id in
• caliper_rubrics_get One rubric with its criteria — each criterion's name, what the judge looks for, and the score scale. Read this to explain a score (which criterion slipped and w
• caliper_rubrics_update Changes a rubric's title, description, visibility, or replaces its criteria wholesale (pass the FULL list — criteria get fresh ids). Evals snapshot the rubric w
• caliper_rubrics_delete Deletes a rubric. Existing evals keep their snapshot of it and keep running; nothing new can bind to it. May return `needs_confirmation`.
• caliper_evals_get One eval: what it targets (Workbench flow + stage, or external), its dataset and rubric ids, the rubric criteria it scores with (the snapshot taken at creation)
• caliper_evals_runs_item_execution What the flow actually DID on one run item: every step in order (what it said, which tools it called with what arguments, what came back) and the final answer,
• caliper_evals_update Changes an eval's title, description, visibility, schedule (interval + time anchor), runOnPublish (queue a run whenever the flow publishes), regressionThreshold
• caliper_evals_delete Deletes an eval and stops its schedule. Run history is kept but no longer reachable from the eval. May return `needs_confirmation`.
• caliper_evals_runs_cancel Stops a run that is still PENDING / RUNNING / SCORING — the brake on a run that's spending more than expected or was started by mistake. The queue stops at once
• caliper_sources_list Apps sending their agent's traces to Caliper — from their own code, OpenTelemetry, or a published Workbench flow — busiest first: name, how it sends, traces in
• caliper_sources_over_time One source day by day: traces, failures, tool calls, lookups that found nothing, tokens, cost, and speed (meanMs; p50/p95 as 'answered within' bucket edges); ea
• caliper_traces_list The traces behind a point on a source's chart, newest first, 50 a page: the question and answer (cut short), tools called, time, tokens, cost, status. Narrow by
• caliper_traces_get One trace: what came in, what went out, and every step in order — model calls (model, tokens, cost, reasoning), tool calls (arguments and result), lookups (quer
• caliper_source_feeds_list Datasets this source keeps adding to as conversations arrive: which dataset, for what (review, eval, spec), the pick it matches, sampling, how many have landed,
• caliper_sources_to_dataset Saves the conversations behind a pick (a day, a tool, a document, failures — the same narrowing as caliper_traces_list) into a new dataset (newDatasetName) or a
• caliper_source_feeds_stop Stops a source from adding new conversations to a dataset. What it already added stays. Get the feedId from caliper_source_feeds_list. May return `needs_confirm
• caliper_starters_list The starter sets Caliper ships: prompt injection, system-prompt leaking, over-refusal, personal-data handling, bias under ambiguity. Each is a small original da
• caliper_starters_install Copies one starter into the workspace as a new dataset and a new rubric the user owns and can edit. Returns both ids; the next step is caliper_evals_create bind
• caliper_datasets_list Lists datasets in the active workspace. Returns summaries (id, name, item count); items are not included.
• caliper_flow_performance How a Workbench flow is ACTUALLY doing, with receipts: every Caliper eval targeting the flow, recent runs with scores, the latest run decomposed into per-criter

Details

• Version: 1.0.0
• Transports: streamable-http
• Uptime (7 days): 100%
• Median latency: 1037 ms
• First indexed: 2026-11-01T08:51:17.799Z
• Last seen in registry: 2026-11-01T08:51:17.799Z
• Website: https://zerowidth.ai/docs/overview/mcp#connect-to-one-product

Badge for your README

Shows this live status and links here; updates hourly.

[![lookupd status](https://lookupd.com/badge/ai.zerowidth/caliper.svg)](https://lookupd.com/servers/ai.zerowidth/caliper)

• 62033 servers
• Last crawl: 2026-11-11T15:38:39.561Z
• API base: https://api.lookupd.com
• MCP URL: https://api.lookupd.com/mcp