Vidlizer

What is Vidlizer?

Vidlizer pulls frames out of any video, image, or PDF using ffmpeg, sends them to a vision LLM, and returns a structured JSON flow array — one entry per scene. Each entry describes what happened, who was on screen, what text was visible, and what changed. It can run fully local via Ollama (no data leaves your machine) or connect to cloud models via OpenRouter.

How to use Vidlizer?

Install with uvx vidlizer, pipx install vidlizer, or pip install vidlizer, then run vidlizer setup to auto-detect providers and write your .env file. After setup, run vidlizer <file> (e.g., vidlizer demo.mp4, vidlizer "https://youtube.com/watch?v=...") to analyze the input. Use --provider and --model flags to choose a specific provider or model, or run with no arguments for an interactive file picker.

Key features of Vidlizer

• Supports local videos, images, PDFs, and URLs (YouTube, Loom, Vimeo, Twitter)
• 4 providers: Ollama (fully offline), LM Studio, oMLX, OpenRouter — auto-detected
• Cross-provider fallback if primary model fails
• Automatic JSON repair for malformed model output
• 3 output formats: JSON (default), Markdown, plain-text summary
• Audio transcription via Apple MLX Whisper merged into each step
• MCP server for use with Claude Code, Cursor, Claude Desktop
• In-memory cache, cost guard (MAX_COST_USD), and live progress indicators

Use cases of Vidlizer

• Analyze video demos or tutorials — get a structured scene-by-scene breakdown
• Extract text and actions from PDFs — parse presentations or documents frame by frame
• Generate documentation from screen recordings — produce step-per-section Markdown or JSON
• Transcribe and annotate podcasts or lectures — combine visual changes with spoken audio
• Batch analyze media archives — use the stats and caching to track usage and cost

FAQ from Vidlizer

Can I use Vidlizer completely offline?

Yes. Set provider to Ollama and it runs entirely locally with no API key, no data leaving your machine. You just need Ollama installed with a vision model pulled (e.g., qwen2.5vl:3b).

Does Vidlizer support YouTube and other URLs?

Yes. You can pass a YouTube, Loom, Vimeo, or Twitter URL directly — Vidlizer will download and analyze the video.

What output formats are available?

Three formats: --format json (default, full structured flow array), --format markdown (step-per-section document), and --format summary (plain text grouped by phase). Default output path is <normalized-name>.analysis.<ext>.

What are the system requirements?

macOS (Apple Silicon recommended), Python 3.10+, ffmpeg (auto-installed via Homebrew), and either Ollama (5+ GB RAM) or LM Studio, oMLX, or an OpenRouter API key.

How does Vidlizer handle model failures or malformed JSON?

It automatically falls back to another provider if the primary model fails. For malformed JSON output, it re-sends the result to the model for repair before skipping the step.