lookupd

io.github.saulius876-lgtm/website-content-crawler

needs login checked

Unclaimed · Unverified

Agent searches it appeared in, last 30 days: 1. Run this server? Claim the listing: verified label, uptime alerts and analytics.

Publisher-provided title

Website to Markdown

Publisher-provided description

Web pages and whole sites to clean Markdown for RAG and AI agents. Crawls links and sitemaps.

Summary by lookupd

Fetches web pages or whole sites, following links and sitemaps, and returns clean Markdown content for retrieval or agent use.

Install

Claude Code, remote (streamable-http)

claude mcp add --transport http website-content-crawler 'https://mcp.apify.com/?tools=sauliusautomatesit/website-content-crawler-api'

Codex, remote (streamable-http)

codex mcp add website-content-crawler --url 'https://mcp.apify.com/?tools=sauliusautomatesit/website-content-crawler-api'

Cursor, remote (streamable-http) → .cursor/mcp.json

{
  "mcpServers": {
    "website-content-crawler": {
      "url": "https://mcp.apify.com/?tools=sauliusautomatesit/website-content-crawler-api"
    }
  }
}

Other clients: docs. Or tell your agent: Set up lookupd for me: read https://lookupd.com/agent.md and follow it.

Details

Badge for your README

Shows this live status and links here; updates hourly.

lookupd status badge for io.github.saulius876-lgtm/website-content-crawler

[![lookupd status](https://lookupd.com/badge/io.github.saulius876-lgtm/website-content-crawler.svg)](https://lookupd.com/servers/io.github.saulius876-lgtm/website-content-crawler)