io.github.77777R7/octocrawl

up checked 2026-11-11T05:38:06.357Z

Unclaimed · Unverified

Publisher-provided title

Octocrawl

Publisher-provided description

Pages, crawls and URL batches as clean Markdown and JSON for AI agents, with an Evidence Record.

Summary by lookupd

Reads web pages and site URL lists as Markdown or JSON with fetch metadata, including status, robots.txt decision and content hashes.

Install

Claude Code, remote (streamable-http)

claude mcp add --transport http octocrawl https://mcp.octocrawl.dev/mcp

Claude Code, remote (streamable-http)

claude mcp add --transport http octocrawl https://mcp.octocrawl.dev/mcp-oauth

Claude Code, npm package (stdio)

claude mcp add octocrawl -- npx -y @octocrawl/mcp

Codex, remote (streamable-http)

codex mcp add octocrawl --url https://mcp.octocrawl.dev/mcp

Codex, remote (streamable-http)

codex mcp add octocrawl --url https://mcp.octocrawl.dev/mcp-oauth

Codex, npm package (stdio)

codex mcp add octocrawl -- npx -y @octocrawl/mcp

Cursor, remote (streamable-http) → .cursor/mcp.json

{
  "mcpServers": {
    "octocrawl": {
      "url": "https://mcp.octocrawl.dev/mcp"
    }
  }
}

Cursor, remote (streamable-http) → .cursor/mcp.json

{
  "mcpServers": {
    "octocrawl": {
      "url": "https://mcp.octocrawl.dev/mcp-oauth"
    }
  }
}

Cursor, npm package (stdio) → .cursor/mcp.json

{
  "mcpServers": {
    "octocrawl": {
      "command": "npx",
      "args": [
        "-y",
        "@octocrawl/mcp"
      ]
    }
  }
}

Other clients: docs. Or tell your agent: Set up lookupd for me: read https://lookupd.com/agent.md and follow it.

Publisher-provided text

scrape Read one URL as Markdown with its Evidence Record (final URL, fetch time, HTTP status, robots.txt decision, hashes), over HTTP first and in the browser when the
map List a site's URLs without reading each page: the start page's links (read over HTTP alone) and the entries of the sitemaps the site declares, within one deadli

Version: 0.8.12
Transports: streamable-http, stdio
Uptime (7 days): 100%
Median latency: 1284 ms
First indexed: 2026-11-01T09:03:23.856Z
Last seen in registry: 2026-11-11T03:17:16.497Z
Repository: https://github.com/77777R7/Octocrawl
Website: https://octocrawl.dev/docs/connect-mcp/?utm_source=mcp-registry&utm_medium=listing

Shows this live status and links here; updates hourly.

[![lookupd status](https://lookupd.com/badge/io.github.77777R7/octocrawl.svg)](https://lookupd.com/servers/io.github.77777R7/octocrawl)