dev.workers.pathwren.www/robots-policy-lint

up checked 2026-11-11T04:38:50.157Z

Unclaimed · Unverified

Agent searches it appeared in, last 30 days: 0. Run this server? Claim the listing: verified label, uptime alerts and analytics.

Publisher-provided title

Robots.txt Policy Lint

Publisher-provided description

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key

Summary by lookupd

Lints and compares robots.txt files for a host and checks whether specific paths are allowed for AI crawler access.

Install

Claude Code, remote (streamable-http)

claude mcp add --transport http robots-policy-lint https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots

Codex, remote (streamable-http)

codex mcp add robots-policy-lint --url https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots

Cursor, remote (streamable-http) → .cursor/mcp.json

{
  "mcpServers": {
    "robots-policy-lint": {
      "url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots"
    }
  }
}

Other clients: docs. Or tell your agent: Set up lookupd for me: read https://lookupd.com/agent.md and follow it.

Tools (9 tools)

Publisher-provided text

no_arguments_lint_this_hosts_robots_txt TAKES NO ARGUMENTS. POST {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} to https://ww
am_i_allowed {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"am_i_allowed","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/ro
whoami {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — POST that to https://www.pathwren.workers.dev/mcp/robots and the answ
example {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/robots
lint_robots_txt FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Parse a r
check_path_allowed FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Evaluate
audit_ai_access FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Evaluate
diff_robots_txt FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Compare t
merge_policy FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Merge one

Details

Version: 1.4.0
Transports: streamable-http
Uptime (7 days): 100%
Median latency: 412 ms
First indexed: 2026-11-01T09:02:02.904Z
Last seen in registry: 2026-11-01T09:02:02.955Z
Website: https://www.pathwren.workers.dev/c/mcp-registry-official/mcp-robots.html

Badge for your README

Shows this live status and links here; updates hourly.

[![lookupd status](https://lookupd.com/badge/dev.workers.pathwren.www/robots-policy-lint.svg)](https://lookupd.com/servers/dev.workers.pathwren.www/robots-policy-lint)

62141 servers
Last crawl: 2026-11-11T18:17:25.333Z
API base: https://api.lookupd.com
MCP URL: https://api.lookupd.com/mcp