Crawlbase Web MCP Server

@crawlbase

Visit Server

3 months ago

Crawlbase Web MCP Server (Model Context Protocol) connects AI agents and large language models (LLMs) with real-time web data. Built on Crawlbase’s proven web scraping and crawling infrastructure, it handles JavaScript rendering, anti-bot protection, and web data extraction at scale — powering Claude, Cursor, and Windsurf. It delivers a reliable, production-ready pipeline of live, structured data for AI workflows and intelligent applications.

Overview Tools Comments

Overview

What is Crawlbase MCP?

Crawlbase MCP is a Model Context Protocol (MCP) server that bridges AI agents and the live web. Instead of relying on outdated training data, your LLMs can now fetch fresh, structured, real-time content — powered by Crawlbase’s proven crawling infrastructure trusted by 70,000+ developers worldwide.

It handles the complexity of scraping for you:

JavaScript rendering for modern web apps
Proxy rotation & anti-bot evasion
Structured outputs (HTML, Markdown, screenshots)

How It Works

Get Free Crawlbase Tokens → Sign up at Crawlbase ↗, get free Normal, and JavaScript tokens.
Add MCP Config → Connect Crawlbase MCP to Claude, Cursor, or Windsurf by updating your mcpServers config.
Start Crawling → Use commands like crawl, crawl_markdown, or crawl_screenshot to bring live web data into your AI agent.

Features

Real-time web scraping for AI agents
JavaScript rendering (SPAs & dynamic pages)
Proxy rotation to bypass blocks & captchas
Structured outputs (HTML, Markdown, Screenshots)
Seamless MCP integration with Claude, Cursor & Windsurf

Use Cases

Research with up-to-date articles & reports
Monitor e-commerce products & prices
Fetch real-time news & financial data
Aggregate content for data pipelines
Power AI agents with fresh, accurate information

Try in Playground

Server Config

{
  "mcpServers": {
    "crawlbase": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "@crawlbase/mcp@latest"
      ],
      "env": {
        "CRAWLBASE_TOKEN": "your_token_here",
        "CRAWLBASE_JS_TOKEN": "your_js_token_here"
      }
    }
  }
}

Build with ShipAny.