MCP Web Extractor

What is MCP Web Extractor?

MCP Web Extractor is a Model Context Protocol (MCP) server that uses Readability.js to fetch web pages and extract the main content, stripping ads, sidebars, and other distractions. It returns clean text along with metadata (title, excerpt, etc.) and is designed for saving readable versions of articles to Obsidian notes.

How to use MCP Web Extractor?

Clone the repository, install dependencies with npm install, build the project with npm run build, and start the server with npm start. The server runs on http://localhost:3000 with the MCP endpoint at http://localhost:3000/mcp. You can use the included client example (ts-node-esm client-example.ts) or integrate with Obsidian via the provided obsidian-integration.ts example.

Key features of MCP Web Extractor

• Extracts readable content from any URL
• Removes ads, sidebars, and other distractions
• Returns clean text along with metadata (title, excerpt, etc.)
• Easy integration with Obsidian via MCP

Use cases of MCP Web Extractor

• Saving clean, readable versions of articles to Obsidian notes
• Extracting main content from web pages for further processing

FAQ from MCP Web Extractor

What technology does MCP Web Extractor use for extraction?

It uses Readability.js to parse and extract the main content from web pages.

How do I install and run MCP Web Extractor?

Clone the repository, run npm install, npm run build, and npm start. The server will start on http://localhost:3000.

What API endpoint does MCP Web Extractor provide?

It provides an extract-content capability that takes a { url: string } parameter and returns { title, content, textContent, excerpt, siteName }.

Can I integrate MCP Web Extractor with Obsidian?

Yes, the repository includes an obsidian-integration.ts file as an example of how to integrate the MCP server with Obsidian.

What license is MCP Web Extractor released under?

It is released under the MIT License.