com.atalaya-ai/ai-crawler-check

up checked 2026-11-11T11:39:09.237Z

Unclaimed · Unverified

Publisher-provided title

Atalaya AI crawler check

Publisher-provided description

Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.

Summary by lookupd

Checks whether named AI crawlers can access and read a public website, and lists the crawler user-agent tokens by vendor and purpose.

Install

Claude Code, remote (streamable-http)

claude mcp add --transport http ai-crawler-check https://atalaya-ai.com/mcp

Codex, remote (streamable-http)

codex mcp add ai-crawler-check --url https://atalaya-ai.com/mcp

Cursor, remote (streamable-http) → .cursor/mcp.json

{
  "mcpServers": {
    "ai-crawler-check": {
      "url": "https://atalaya-ai.com/mcp"
    }
  }
}

Other clients: docs. Or tell your agent: Set up lookupd for me: read https://lookupd.com/agent.md and follow it.

Publisher-provided text

check_ai_crawler_access Checks whether AI crawlers can reach and read a public website or web app: which AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Goo
list_ai_crawlers Lists the AI crawler user-agent tokens Atalaya checks, by vendor and purpose (training, search index, user-triggered fetch), so you can write a robots.txt that

Shows this live status and links here; updates hourly.

[![lookupd status](https://lookupd.com/badge/com.atalaya-ai/ai-crawler-check.svg)](https://lookupd.com/servers/com.atalaya-ai/ai-crawler-check)