lookupd

io.github.jmshinhwa/robots-txt-ai-crawler-audit

not probeable (local package)

Unclaimed · Unverified

Agent searches it appeared in, last 30 days: 0. Run this server? Claim the listing: verified label, uptime alerts and analytics.

Publisher-provided description

Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI

Summary by lookupd

Fetches a site's robots.txt and reports, line by line, which AI crawler user-agent tokens are allowed or blocked for training versus other…

Install

Claude Code, npm package (stdio)

claude mcp add robots-txt-ai-crawler-audit -- npx -y @readystack/robots-txt-ai-crawler-audit

Codex, npm package (stdio)

codex mcp add robots-txt-ai-crawler-audit -- npx -y @readystack/robots-txt-ai-crawler-audit

Cursor, npm package (stdio) → .cursor/mcp.json

{
  "mcpServers": {
    "robots-txt-ai-crawler-audit": {
      "command": "npx",
      "args": [
        "-y",
        "@readystack/robots-txt-ai-crawler-audit"
      ]
    }
  }
}

Other clients: docs. Or tell your agent: Set up lookupd for me: read https://lookupd.com/agent.md and follow it.

Details

Badge for your README

Shows this live status and links here; updates hourly.

lookupd status badge for io.github.jmshinhwa/robots-txt-ai-crawler-audit

[![lookupd status](https://lookupd.com/badge/io.github.jmshinhwa/robots-txt-ai-crawler-audit.svg)](https://lookupd.com/servers/io.github.jmshinhwa/robots-txt-ai-crawler-audit)