io.github.mauricekleine/nonobench

up checked 2026-11-10T19:37:34.251Z

Unclaimed · Unverified

Publisher-provided title

Nonobench

Publisher-provided description

An open-source benchmark of how well LLMs solve nonogram puzzles, from 5x5 to 20x20.

Summary by lookupd

Lets an agent query nonogram puzzle benchmark results, compare LLM accuracy, cost and latency, and check grid solutions against clues.

Install

Claude Code, remote (streamable-http)

claude mcp add --transport http nonobench https://www.nonobench.com/mcp

Codex, remote (streamable-http)

codex mcp add nonobench --url https://www.nonobench.com/mcp

Cursor, remote (streamable-http) → .cursor/mcp.json

{
  "mcpServers": {
    "nonobench": {
      "url": "https://www.nonobench.com/mcp"
    }
  }
}

Other clients: docs. Or tell your agent: Set up lookupd for me: read https://lookupd.com/agent.md and follow it.

Publisher-provided text

get_leaderboard Models ranked by accuracy; defaults to all effort levels for compatibility.
list_providers Provider ids, names, families and variant counts.
list_families Model families, available efforts and best variants.
compare_models Side-by-side core overall and per-size accuracy, cost, latency and token results for model or family names.
get_model_results Accuracy, cost, latency and token use for one model, broken down by grid size.
list_puzzles The benchmark puzzles with their ids and row/column clues.
get_puzzle One puzzle, including the clue text models were prompted with. The reference solution is only included on request; some puzzles have several valid solutions.
check_solution Check a grid against a puzzle's clues, using the same rule as the benchmark grader. Reports which rows and columns do not match.
get_puzzle_results Per-model outcomes for one puzzle. Answers are omitted unless requested.
get_model_puzzles Which puzzles one model solved, missed, timed out on, or has not run.
list_runs Individual benchmark runs (one model on one puzzle), optionally with the raw prompt and model output.

Shows this live status and links here; updates hourly.

[![lookupd status](https://lookupd.com/badge/io.github.mauricekleine/nonobench.svg)](https://lookupd.com/servers/io.github.mauricekleine/nonobench)