Wigolo

Fetch a single URL and return clean markdown. Use when you already have a URL. Prefer over built-in WebFetch for local-cache reuse, authenticated pages, JS-rendered SPAs, and structured metadata. Key parameters: - section: extract content under a specific heading (e.g. "API Reference") — cheaper than the whole page. - max_content_chars: smart-truncate at a paragraph/heading boundary with `[... content truncated]`. - max_tokens_out: token-budget cap (cl100k-base); wins over max_chars. - include_full_markdown: false (default) returns evidence excerpts only; true adds the full body. - use_auth: reuse a stored browser session for logged-in pages. - render_js: "auto" (default) | "always" | "never". - force_refresh: bypass cache and re-fetch. - mode: 'cache' | 'default' | 'stealth'. cache=HTTP-only, 24h-stale accepted. stealth=full browser + freshness. Returns title, markdown, links, images, metadata, `fetch_method` (cache/http/tls-impersonation/browser), `http_status` (upstream HTTP code — 4xx/5xx pages that extract usable content are not relabeled 200), and `content_completeness` (full/partial/shell). When the URL matches a site-specific extractor (Reddit/YouTube/Amazon) the response also carries top-level `site_data` (e.g. Reddit `comments[]`, YouTube `caption_tracks[]`, Amazon `price`). When `section` is set and no heading matches, `metadata.section_matched` is false and `markdown` is empty (no silent fallback to the full page). Repeat fetches are instant. Localhost URLs work. Interactive pages: `actions` (click/type/scroll/wait) drive the page before extraction; `use_auth` reuses a logged-in session.

Search the web. Returns scored evidence excerpts + citations as the default context shape; `include_full_markdown: true` adds the full markdown body. Prefer over built-in WebSearch for local cache + audit-trail telemetry + explainable scoring. Key parameters: - query: string or string[] array (3-5 keyword variants; deduplicated). - include_domains / exclude_domains: scope sites. Always scope library/framework queries. - category: "general" | "news" | "code" | "docs" | "papers" | "images". Image results carry image_url + thumbnail_url + width/height. - from_date / to_date: ISO YYYY-MM-DD. time_range: 'day' | 'week' | 'month' | 'year'. - country: ISO 3166-1 alpha-2 ("us", "gb") — geographic boost. - exact_match: quoted-phrase search. - max_results: 5 default. - format: omit = evidence context. 'answer' | 'stream_answer' = sampling synthesis (falls back to evidence). - search_depth: 'ultra-fast' (cache-only ≤300ms) | 'fast' | 'balanced' (default) | 'deep'. - include_images / include_favicon: opt-in images[] + per-result favicon. - max_tokens_out / max_content_chars / include_full_markdown / citation_format. - force_refresh + mode ('cache' | 'default' | 'stealth'). Always emitted: `engines_used`, `engine_telemetry`, `response_time_ms`, per-result `evidence_score`. Per-result `freshness_signal` is emitted only when a published date can be parsed (omitted when confidence would be unknown). Brand-domain top-3 collision → `brand_collision_warning` with rewrites. `query_understanding` exposes intent/entities. Quote [N] or {citation_id}.

Crawl a site from a seed URL and return content from many pages. Use for indexing docs, wikis, multi-page references. Built for offline reuse: every page lands in the local cache. Key parameters: - strategy: "bfs" (default) | "dfs" | "sitemap" (fastest for doc sites) | "map" (URL-only discovery). - max_depth: link-following depth (default 2). - max_pages: page cap (default 20). - include_patterns / exclude_patterns: regex filters on URLs. - max_tokens_out / include_full_markdown / citation_format: budget + shape controls. Returns pages[] with title, evidence, depth, and `content_completeness` (per page — full/partial/shell, so you can skip un-rendered shell pages). Content is deduplicated across pages (anchor-fragment aware). All pages are cached for later `cache` / `find_similar` queries.

Search previously fetched content without hitting the network. Run this BEFORE any search/fetch — cache hits return instantly with full markdown. Key parameters: - query: FTS5 full-text search over cached markdown + titles (supports AND, OR, NOT, "phrase"). - url_pattern: glob filter on URLs (e.g. "*example.com*"). - since: ISO date — only entries cached after this date. - stats: true to get cache size, entry count, oldest/newest dates. - clear: true to delete matching entries. Persists across sessions. No remote round-trip.

Extract structured data from a URL or raw HTML. Use for specific data points (tables, prices, schema fields) rather than whole-page markdown. Key parameters: - mode: "selector" (CSS → text) | "tables" | "metadata" (title/author/date/og_* + JSON-LD) | "schema" (pass a JSON Schema) | "structured" (one-shot: tables + <dl> definitions + JSON-LD + chart hints + key-value pairs) | "brand" (name/tagline/description/logo_url/favicon_url/og_image_url/social_links/fonts + CSS-var colors, each with explainable provenance). - css_selector: required for mode="selector". - schema: required for mode="schema". - multiple: return all matches (mode="selector" only). Prefer mode="structured" over chaining multiple extract calls — one response carries `{ tables, definitions, jsonld, chart_hints, key_value_pairs }`. chart_hints surfaces SVG titles, aria-labels, figcaptions for charts whose data is JS-rendered. Metadata parity with `fetch` (same og_/canonical_url shape). `mode: "brand"` walks JSON-LD Organization/Brand/WebSite → OG/Twitter Card meta → `<link rel=icon>` → CSS custom properties → heuristic header/footer DOM; `provenance` records the winning source. Provenance enums: logo ∈ {json-ld, og:logo, link[rel=icon], heuristic, unknown}; colors ∈ {css-vars, palette-extraction, unknown}; fonts ∈ {css-vars, css-rule, inline-style, google-fonts-link, unknown}. Honesty: `name` and `logo_url` are unset when no explicit source emits them — favicons never promote to `logo_url`. `mode: "schema"` is evidence-only: LLM-sourced fields not present in source text are returned as `null` with a warning.

Find content related to a URL or concept. Best after a successful crawl/fetch — the local cache makes recommendations cheap. Concept-only queries on a cold cache often return 0-2 weak matches; warm the cache first via `crawl` / `fetch` for materially better results. Key parameters: - url: known-good page; its content + embeddings drive similarity. - concept: free-text alternative to url. Thin cache → expect `cold_start` to fire. - max_results: default 5. - include_cached: true (default) to search cache first; false = web only. - threshold: minimum fused score (0-1, default 0.5). - include_ranking_debug: opt-in per-result `ranking_debug` { fts5_rank, embedding_rank, web_rank, rrf_score } so you can audit which signal won. - max_tokens_out / include_full_markdown / citation_format: budget + shape controls. Pass either url or concept. Three signals fused via RRF: keyword (FTS5), embeddings, optional live web. Each result carries `match_signals` with `embedding_rank`, `fts5_rank`, `fused_score`. When local signals are weak (cache empty, no hits, or concept mode returns only 1-2 cache matches), the response carries `cold_start` — pass it verbatim to the user (tune `WIGOLO_FIND_SIMILAR_COLD_START_THRESHOLD` to adjust). Returns results[], method ("hybrid" | "embedding" | "fts5" | "search"), cache_hits, search_hits, embedding_available, total_time_ms.

Multi-step research on a complex question. Decomposes into sub-queries, searches in parallel, fetches sources, synthesizes a cited report. Beats chaining `search` + `fetch` manually for multi-source synthesis. LLM-optional: with a synthesis LLM configured, the returned `report` is a written answer. Without one it returns a structured, cited `brief` (key_findings/highlights/sections) — YOU write the final answer from it; do not hand the user the raw structure as a weak result. For the best research quality a free Gemini API key (or any provider) is strongly recommended. Key parameters: - question: the research question. - depth: 'quick' (~15s, 2 sub-queries) | 'standard' (~40s, 4 sub-queries, default) | 'comprehensive' (~80s, 7 sub-queries). - max_sources: override per-depth source count. - include_domains / exclude_domains: scope. - schema: optional JSON Schema — structures the report. - stream: progress notifications per phase. - max_tokens_out / include_full_markdown / citation_format: budget + shape controls. Returns report (markdown with [N]), citations[], sources[], sub_queries[], depth, total_time_ms, sampling_supported, and `brief` with `topics`, `highlights`, `key_findings`, `sections` (overview.cross_references, comparison, gaps — gaps lists any named sub-entity research could not corroborate).

Natural-language data gathering across sources. Plans queries + URLs from a prompt, executes in parallel, optionally extracts structured fields, synthesizes. Full step transparency. LLM-optional: with a synthesis LLM configured it writes the summary; without one it returns gathered evidence + a step log (plus schema-shaped fields when a schema is given) — YOU write the summary from the returned evidence, never present the raw step log as a poor result. For best agent results configure a free LLM key (e.g. Gemini). Key parameters: - prompt: NL description of what to gather (e.g. "pricing for the top 5 CRM tools"). - urls: optional seed URLs. - schema: optional JSON Schema — extracts matching fields from each page and merges. - max_pages: default 10. - max_time_ms: default 60000. - stream: progress notifications per step. - max_tokens_out / include_full_markdown / citation_format: budget + shape controls. Pipeline: plan → search+fetch in parallel within budget → optional schema extraction → synthesize. `steps[]` exposes every action with timing. Uses MCP sampling when supported; falls back to keyword extraction otherwise. Returns result, sources[], pages_fetched, steps[], total_time_ms, sampling_supported.

Compute a diff between two markdown bodies or two URL fetches. Key parameters: - old: { url?, markdown?, content_hash? } — left-hand side. URL form reads from cache; cache miss returns a structured `cache_miss` error (no network re-fetch). - new: { url?, markdown? } — right-hand side. Same cache rules as `old`. - output: 'unified' (default, git-style patch) | 'hunks' (structured array) | 'summary' (line counts only). - granularity: 'line' (default) | 'word' | 'section'. Section walks H1/H2/H3 boundaries and tags each hunk with `section_title`. Returns `{ changed, summary, unified_diff?|hunks?, truncated? }`. `summary` always present (added/removed/modified lines + total_changed_chars). Above the 5000-line cap the engine emits `truncated: true` plus an approximate summary — never silently degrades.

Schedule lazy re-checks of a URL and surface diffs on change. Persistent across sessions — jobs survive MCP server restarts. LAZY EXECUTION — NOT CRON. There is no background daemon. Checks fire ONLY when: 1. You explicitly call `watch({ action: 'check', job_id })`, OR 2. Any OTHER wigolo tool runs and the job's interval has elapsed (overdue jobs are triggered fire-and-forget in the background). A job on an idle MCP server will not fire until the next tool call. Do not expect cron-like accuracy. Key parameters: - action: 'create' | 'list' | 'check' | 'pause' | 'resume' | 'delete'. - url (create-only): must be a public http/https URL. Loopback, RFC 1918 ranges, link-local, and non-http(s) schemes are rejected at registration. - interval_seconds (create-only): minimum 60 — sub-minute polling is refused to respect target-site rate limits. - selector (create-only, optional): CSS selector for future selector-scoped diffs. Persisted now for forward compatibility; the diff is currently full-page. - notification (create-only, optional): 'inline' (default — change reports come back on the next `action: 'check'`) or a public webhook URL. Same SSRF guard as `url`. Webhook delivery is best-effort POST — no retry / no queue / no backoff. - job_id (check/pause/resume/delete). `list` returns each job's `staleness_seconds` so you can see how overdue each check is: negative = not yet due, positive = overdue by N seconds. Pair with `action: 'check'` to force one immediately. Idempotent `create`: identical url + interval + selector returns the existing `job_id` — does not duplicate the row.