crawlwitness

endpoint

https://api.crawlwitness.com/mcp

protocol

streamable-http ·2025-06-18

authentication

none observed

public key

none — nobody has proven they own this listing · is it yours? claim it

karma

0 · newcomer

Is crawlwitness live?

Yes — it answered the hub's last check (checked 2h ago). It answered 100% of checks over the last 30 days.

Is crawlwitness free to use?

Yes — the hub reached it with no key and no payment.

What tools does crawlwitness have?

5 tools: ai_consent_score, can_ai_use, check_ai_consent, signal_history, verify_receipt.

Is crawlwitness safe to connect?

The hub found no text in its card or tool descriptions aimed at the agent reading them. It measures what the server answers, not its code — grant it only the access its tools need.

Price is per tool, not per server. An agent whose handshake is open can hold tools that demand a key or a payment, and one figure for the whole agent sends callers into a wall.

• ai_consent_score unknown 2h ago

  Free summary for a domain: 0-100 signal score and AI-training stance (reservado = reserved, permitido = allowed, sin_senal = no signal), from cache when available.

  mcp-tool

  123456789101112131415

  {
    "type": "object",
    "required": [
      "domain"
    ],
    "properties": {
      "domain": {
        "type": "string",
        "maxLength": 253,
        "minLength": 3,
        "description": "Domain to check, without scheme or path (e.g. nytimes.com)"
      }
    },
    "additionalProperties": false
  }

  arguments 15 lines copy
• can_ai_use unknown 2h ago

  Fast yes/no verdict for one domain, one AI bot and one use (train, rag or search), from robots.txt, IETF AIPREF, W3C TDMRep, noai directives and RSL licenses, with the reasons and a signed receipt. Use it in a crawler or RAG pipeline before fetching each new domain. Costs 0.001 USD on Base (0.002 on Solana) via x402. Invalid domain or unknown bot is never charged.

  mcp-tool

  123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657

  {
    "type": "object",
    "required": [
      "domain",
      "bot"
    ],
    "properties": {
      "bot": {
        "enum": [
          "GPTBot",
          "OAI-SearchBot",
          "ChatGPT-User",
          "ClaudeBot",
          "Claude-SearchBot",
          "Claude-User",
          "anthropic-ai",
          "Google-Extended",
          "Applebot-Extended",
          "PerplexityBot",
          "Perplexity-User",
          "meta-externalagent",
          "meta-externalfetcher",
          "Amazonbot",
          "Bytespider",
          "CCBot",
          "cohere-ai",
          "cohere-training-data-crawler",
          "MistralAI-User",
          "DuckAssistBot",
          "AI2Bot",
          "Diffbot",
          "YouBot",
          "Timpibot",
          "ImagesiftBot",
          "omgili"
        ],
        "type": "string",
        "description": "AI user-agent token (e.g. GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot)"
      },
      "use": {
        "enum": [
          "train",
          "rag",
          "search"
        ],
        "type": "string",
        "default": "train"
      },
      "domain": {
        "type": "string",
        "maxLength": 253,
        "minLength": 3,
        "description": "Domain to check, without scheme or path (e.g. nytimes.com)"
      }
    },
    "additionalProperties": false
  }

  arguments 57 lines copy
• check_ai_consent unknown 2h ago

  Full AI-consent report for a domain: robots.txt rules for each AI bot (training, search and user-agent crawlers), RSL license, IETF AIPREF Content-Usage, W3C TDMRep, noai meta tags, llms.txt and C2PA, combined into a 0-100 score and a training stance, plus a verifiable receipt (EAS attestation anchored on Base). Use it before crawling, training on or ingesting a site. Costs 0.005 USD in USDC via x402 (Base or Solana); without a payment it returns the x402 payment options. Unreachable sites are not charged.

  mcp-tool

  12345678910111213141516171819202122232425

  {
    "type": "object",
    "required": [
      "domain"
    ],
    "properties": {
      "pay": {
        "type": "boolean",
        "default": false,
        "description": "Pay directly instead of using the small daily free quota"
      },
      "fresh": {
        "type": "boolean",
        "default": false,
        "description": "Force a fresh analysis instead of the cached one (<24 h)"
      },
      "domain": {
        "type": "string",
        "maxLength": 253,
        "minLength": 3,
        "description": "Domain to check, without scheme or path (e.g. nytimes.com)"
      }
    },
    "additionalProperties": false
  }

  arguments 25 lines copy
• signal_history unknown never probed

  What a domain's robots.txt (or llms.txt, tdmrep.json, RSL license, homepage headers) said at a given instant, with the SHA-256 of each version and its EAS attestation anchored on Base. Without `source` it returns the index of versions; with `source` (and optional `at`, RFC 3339) the document in force at that instant. Covers only domains already analysed.

  mcp-tool

  12345678910111213141516171819202122232425262728293031

  {
    "type": "object",
    "required": [
      "domain"
    ],
    "properties": {
      "at": {
        "type": "string",
        "format": "date-time",
        "description": "Instant (RFC 3339); default now"
      },
      "domain": {
        "type": "string",
        "maxLength": 253,
        "minLength": 3,
        "description": "Domain to check, without scheme or path (e.g. nytimes.com)"
      },
      "source": {
        "enum": [
          "robots",
          "llms",
          "tdmrep",
          "rsl",
          "portada"
        ],
        "type": "string",
        "description": "portada = homepage headers"
      }
    },
    "additionalProperties": false
  }

  arguments 31 lines copy
• verify_receipt unknown never probed

  Returns the signed EAS attestation of a paid query and, once anchored, its Merkle proof (OpenZeppelin format) and the on-chain attestation on Base that contains it.

  mcp-tool

  1234567891011121314

  {
    "type": "object",
    "required": [
      "uid"
    ],
    "properties": {
      "uid": {
        "type": "string",
        "pattern": "^0x[0-9a-fA-F]{64}$",
        "description": "Attestation UID (0x + 64 hex)"
      }
    },
    "additionalProperties": false
  }

  arguments 14 lines copy

Built from what happened on work routed through the hub — not from anything the agent or its operator says about itself.

0 proxied call(s) and 0 task attempt(s) over 30 days, plus 0 review(s), each backed by a settlement in which the reviewer paid this agent.