tech.ratlabs/rat-radar

up checked 2026-11-11T12:37:32.121Z

Unclaimed · Unverified

Publisher-provided title

Rat Radar

Publisher-provided description

Ask before you scrape: is a site open, may I fetch this URL, and how do I extract a page. No key.

Summary by lookupd

Checks whether a site or URL may be scraped per robots.txt, rates site hostility, and recommends extraction approaches and steps for a page.

Install

Claude Code, remote (streamable-http)

claude mcp add --transport http rat-radar https://rat-radar.github-a18.workers.dev/mcp

Codex, remote (streamable-http)

codex mcp add rat-radar --url https://rat-radar.github-a18.workers.dev/mcp

Cursor, remote (streamable-http) → .cursor/mcp.json

{
  "mcpServers": {
    "rat-radar": {
      "url": "https://rat-radar.github-a18.workers.dev/mcp"
    }
  }
}

Other clients: docs. Or tell your agent: Set up lookupd for me: read https://lookupd.com/agent.md and follow it.

Publisher-provided text

check_site Can I scrape this site, and how? Returns a verdict (open, guarded, hostile, blocked), a hostility score, a recommended approach with steps and risks, bot protec
get_playbook What should I build with this site, and how? Give a domain and, optionally, a goal in plain words (for example "price alerts", "sales leads", "new jobs", "chang
list_sites List the sites in the dataset, optionally by category or verdict. Use it to find easy targets or to see what is covered.
xray_page How do I extract the data from THIS page? Fetches one public page and returns a ranked extraction plan: embedded JSON, JSON-LD, repeated records with CSS select
xray_diagram Draw a page's anatomy as an Excalidraw diagram: its regions, the ranked ways to get its data, proof rows, the best route and what the page is made of. Returns a
search_sites Find sites in the dataset by a word: a name, host, category (jobs, shop, news, gov), country or verdict. Use it to discover what is covered before calling check
can_fetch Does robots.txt allow fetching this exact URL? Fetches the live robots.txt and applies the longest-match rule for a user agent (default: * plus GPTBot, ClaudeBo
summary Headline numbers across the dataset: verdict mix, how many sites block AI crawlers (and which), protection vendors, by category and country.

Shows this live status and links here; updates hourly.

[![lookupd status](https://lookupd.com/badge/tech.ratlabs/rat-radar.svg)](https://lookupd.com/servers/tech.ratlabs/rat-radar)