io.github.emmanuelorta/crawlcheck

up checked 2026-11-11T04:38:06.758Z

Unclaimed · Unverified

Agent searches it appeared in, last 30 days: 0. Run this server? Claim the listing: verified label, uptime alerts and analytics.

Publisher-provided title

CrawlCheck

Publisher-provided description

Verification layer for the agentic web: a signed answer about any site before an agent acts.

Summary by lookupd

Lets an agent verify a domain before reading, citing or transacting: safety checks, signed allow/block decisions, robots rules, and…

Install

Claude Code, remote (streamable-http)

claude mcp add --transport http crawlcheck https://crawlcheck.io/mcp

Codex, remote (streamable-http)

codex mcp add crawlcheck --url https://crawlcheck.io/mcp

Cursor, remote (streamable-http) → .cursor/mcp.json

{
  "mcpServers": {
    "crawlcheck": {
      "url": "https://crawlcheck.io/mcp"
    }
  }
}

Other clients: docs. Or tell your agent: Set up lookupd for me: read https://lookupd.com/agent.md and follow it.

Tools (28 tools)

Publisher-provided text

preflight Call BEFORE reading, citing, connecting to or transacting with a domain. Returns a signed decision: allow, warn, require_confirmation, block or unsupported, wit
best_source Call when choosing WHICH business or site to cite, recommend or call. Give domains (2-10 candidate sites you found) or trade and city for a measured market. Ret
resolve_domain Call before fetching from, citing, connecting to or transacting with a domain. Returns one signed answer: what the domain claims, what it actually serves each c
field_check Call before using a phone number, address, opening hours or service area you hold for a business. One signed verdict per business field. field=phone|address|hou
scan_domain Scan a public domain as several crawler identities and return the graded record: grade, AI-visibility score, per-section scores, findings with evidence, and whe
list_verified_capabilities What a domain declares agents can do (OpenAPI operations, MCP tools, A2A skills, API catalog entries), each classed by safety (read_only_public up to financial)
mcp_servers Which remote MCP servers the official MCP Registry lists on a domain, and what each endpoint actually answered when CrawlCheck sent the MCP handshake (initializ
verify_crawler_log Given raw web-server access-log lines, decide for each line that names a crawler (GPTBot, ClaudeBot, Googlebot, PerplexityBot...) whether the source IP falls in
corpus_state Coverage of CrawlCheck's public dataset: how many distinct domains have been measured and how they split by platform, rendering, size, language and kind, with t
resolve_robots Resolve every named answer engine, search index and training crawler against a domain's robots.txt the way a crawler does: most-specific group only, longest mat
resolve_rights What a site declares AI may do with a page, one answer per use (train, ground, summarize, cite, transact), merged from robots.txt read per agent, Content-Signal
check_site_safety Before an agent reads, cites, links to or pays a site: whether it is listed as malware or phishing (Google Safe Browsing, the Cloudflare security resolver, Phis
draft_llms Draft an llms.txt from the domain's own homepage and up to 25 declared pages, using their titles and descriptions. A draft for the owner to cut down, not a publ
fix_robots The domain's served robots.txt, corrected: * group rules copied into named groups that lacked them (shadowing), a Sitemap line added when missing, a minimal rep
fix_entitymap A starter entitymap.json built from the domain's latest scan record: name, phone, coordinates and declared service areas as entities with SERVES relations. Fiel
video_check One YouTube video against the video anchor model: metadata, chapters, captions and storyboard where observed, twelve checks each naming what it reads.
video_channel Every upload on a YouTube channel, read 50 at a time, sorted by views. The upload list is free; the per-check tally needs a Watch licence and is not offered her
video_site The site half of video: homepage plus up to 60 sitemap pages read for YouTube embeds, facades and page-builder widgets, VideoObject nodes and their required pro
registry_lookup Whether a domain is in CrawlCheck's registry and what it serves: llms.txt, agents.md, a media kit at /.well-known/media-kit.json, a reciprocity-tested entity gr
crawler_path Where each crawler's path into a domain dies, as an observed data flow: edge decision, robots.txt as a file, robots.txt rules, then the page, JSON-LD, sitemap,
public_counts The figures CrawlCheck quotes about itself, read from the source: sites_measured, domains_in_corpus, crawler_visits (a rolling window), sections, sections_score
telemetry Verified-crawler traffic observed at crawlcheck.io itself: which named agents arrived, how many claims were confirmed against operator ranges, how many were for
fix_mostly_code A fix plan for PAGE_IS_MOSTLY_CODE: the homepage's measured bytes by category, the largest blocks with what to do with each, ordered steps with the text ratio a
fix_stale_cache A fix plan for STALE_CACHE_SERVED: the homepage fetched now, its Age, the cache layers that name themselves in the headers in purge order (innermost first), the
fix_machine_chain A fix plan for MACHINE_CHAIN_HEAVY: the files a crawler reads before the page and the 404s absent agent files return, then measured reductions (llms.txt to an i
explain_finding Why CrawlCheck decided a finding: the rule and its revision, the exact fetches it compared (identity, status, bytes, sha256), when the condition held on the sit
machine_record The unified machine record for one observation: subject, grade, findings (the top one in full without a licence), capabilities with verification, and the eviden
mcp_precheck For MCP server publishers, before releasing a new version. Give the server name (as listed in the MCP Registry) or its endpoint, the new tools/list result and t

Details

Version: 1.1.0
Transports: streamable-http
Uptime (7 days): 100%
Median latency: 1091 ms
First indexed: 2026-11-01T09:29:08.565Z
Last seen in registry: 2026-11-01T09:29:08.575Z
Repository: https://github.com/emmanuelorta/crawlcheck-core
Website: https://crawlcheck.io

Badge for your README

Shows this live status and links here; updates hourly.

[![lookupd status](https://lookupd.com/badge/io.github.emmanuelorta/crawlcheck.svg)](https://lookupd.com/servers/io.github.emmanuelorta/crawlcheck)

62071 servers
Last crawl: 2026-11-11T16:39:06.253Z
API base: https://api.lookupd.com
MCP URL: https://api.lookupd.com/mcp