#crawler

Official Firecrawl MCP Server - Adds powerful web scraping to Cursor, Claude and any other LLM clients.

One API for Search, Crawling, and Sitemaps

MCP server for podcast discovery through web crawling

MCP (Model Context Protocol) server wrapping Crawl4AI library as functions using Python

mcp server for craw the interview questions in discussion

Turn any website into LLM-ready, and provide SERP

A powerful Model Context Protocol (MCP) server that intelligently fetches and processes web content with nested URL crawling capabilities. Transform any documentation site or web resource into clean, structured markdown files perfect for AI consumption and analysis.

A lightweight MCP server for web searching via DuckDuckGo and extracting page content as Markdown.

This project provides a toolset to crawl websites wikis, tool/library documentions and generate Markdown documentation, and make that documentation searchable via a Model Context Protocol (MCP) server, designed for integration with tools like Cursor.

crawler-mcp-server

An MCP Server that's also an MCP Client. Useful for letting Claude develop and test MCPs without needing to reset the application.

A powerful MCP server extension providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling AI assistants to search the web and extract webpage content programmatically.

A powerful MCP server extension providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling AI assistants to search the web and extract webpage content programmatically.

Bridge the gap between your web crawler and AI language models using Model Context Protocol (MCP). With mcp-server-webcrawl, your AI client filters and analyzes web content under your direction or autonomously, extracting insights from your web content. Support for WARC, wget, InterroBot, Katana, and SiteOne crawlers is available out of the gate. The server includes a full-text search interface with boolean support, resource filtering by type, HTTP status, and more.

Model Context Protocol (MCP) Server for Graphlit Platform

Web Content Retrieval (full webpage, filtered content, or Markdown-converted), Custom User-Agent, Multi-HTTP Method Support (GET/POST/PUT/DELETE/PATCH), LLM-Controlled Request Headers, LLM-Accessible Response Headers, and more.