Turn a long video into a useful written report with the important ideas, visual context, and timestamped evidence preserved.
VideoLens is a broad video-to-report tool with a simple YouTube-first web experience. Paste a link, choose a report style, and get a detailed artifact you can read, search, save, or question later. Unlike transcript-only summarizers, VideoLens combines speech, sampled frames, and on-screen text before producing a professional standalone HTML report, print-quality PDF, Markdown, and machine-readable JSON. Local files and other supported video URLs work too.
View the no-key sample report · Analyze a YouTube video
videolens analyze "https://www.youtube.com/watch?v=..." \
--mode general \
--prompt "Write a detailed report with the main ideas, evidence, and takeaways."Or launch the web UI:
videolens uiMost video summarizers stop at the transcript. That misses slides, diagrams, demonstrations, code, charts, and other information carried visually. VideoLens builds a time-aligned record of what was said and shown, then writes a structured report with citations back to the important moments. A cached intermediate timeline also lets people and agents ask more questions without repeating the expensive extraction.
v0.1 — alpha. End-to-end working: local files + direct URLs + YouTube → timeline → mode-driven analysis → HTML/PDF/Markdown/JSON reports. UI included. Not production-tested at scale.
- Multi-source input — local files (
.mp4/.mov/.webm/.mkv), direct video URLs, YouTube - Cloud-only pipeline — OpenAI for transcription, frame description (+ OCR), and synthesis
- Timestamped timeline — every finding cites evidence at a specific second of the video
- Four report starting points in the web app: Detailed Report, Key Insights, Tutorial Guide, and Interview / Podcast
- Professional report export — standalone branded HTML and a matching print-quality PDF
- Nine underlying analysis modes for specialized use:
general,bug,meeting,ux,tutorial,product_demo,content,privacy, andproduction_recipe - Local web UI (Streamlit) — drag-and-drop upload, mode/frame sliders, tabbed results (Report / Timeline / Frames / Transcript / Cache)
- Per-step caching — re-runs are cheap; per-prompt analysis cache means "ask many questions of one extraction" is free after the first
- Cost-aware frame budgeting —
--max-framescaps vision-API calls; adaptive interval
Source URL or file
│
▼
┌──────────┐ classifies local_file / youtube / direct_url / webpage,
│ resolver │ reports limitations clearly
└──────────┘
│
▼
┌────────────┐ yt-dlp (when remote), ffprobe metadata, ffmpeg audio
│ processors │ extraction + 30 s chunking, ffmpeg frame sampling,
└────────────┘ OpenAI transcription (parallel per chunk), OpenAI vision
│ frame describe + OCR combined (parallel per frame)
▼
┌──────────┐ merges frame summaries + transcript into time-windowed
│ timeline │ segments; per-segment OCR, visual, transcript, confidence
└──────────┘
│
▼
┌──────────┐ one GPT-5.5 call with timeline + user prompt + mode
│ analysis │ guidance → structured Analysis (summary, findings with
└──────────┘ timestamped evidence, recommendations, tasks, limitations)
│
▼
┌──────────┐
│ outputs │ report.html + report.md + analysis.json + cached artifacts
└──────────┘
Each step is cached at .videolens/cache/<source-hash>/ so re-runs are cheap. Cache key includes source content (hashed) plus extraction settings (frame interval, max frames).
Requires Python 3.12+ and ffmpeg + ffprobe on PATH.
# Install ffmpeg (macOS)
brew install ffmpeg
# Clone and install with uv
git clone https://github.com/shadoprizm/videolens.git
cd videolens
uv sync --extra uiSet your OpenAI API key:
export OPENAI_API_KEY=sk-...(The web UI also accepts the key as a password-style input if you'd rather not put it in your environment.)
uv run videolens uiOpens http://localhost:8501. Drop a video, set a prompt and mode, hit Analyze.
Vercel runs the root FastAPI entrypoint declared in pyproject.toml:
[tool.vercel]
entrypoint = "app:app"The Vercel deployment exposes a lightweight HTTP surface at / and /api/health.
The full upload/analyze Streamlit UI remains a local app launched with uv run videolens ui.
Keeping the UI dependencies in the optional ui extra prevents Vercel's Python
function bundle from pulling in Streamlit, pandas, pyarrow, and PDF rendering packages.
Railway is the recommended hosted path for the full Streamlit UI. The repo includes:
Dockerfile— installs Python dependencies,ffmpeg, and PDF-rendering system librariesrailway.json— tells Railway to use the Dockerfile and health-check Streamlit.dockerignore— keeps local caches, videos, and virtualenvs out of the image
In Railway:
- Create a new project from
github.com/shadoprizm/videolens. - Deploy. Railway will build with the root
Dockerfile. - In Networking, generate a public domain with target port
8501.
Do not set a shared OPENAI_API_KEY for the hosted app. VideoLens is BYOK:
users paste their own OpenAI key into the UI, use the tool, and the key is kept
only in that browser session.
Optional: add a Railway volume mounted at /app/.videolens if you want uploads
and processed-video cache files to survive redeploys.
uv run videolens analyze <source> --prompt "<your question>" [--mode MODE]Useful flags:
| Flag | Default | Purpose |
|---|---|---|
--mode, -m |
general |
Analysis mode |
--max-frames |
40 |
Hard cap on frames sent to the vision model (cost dial) |
--frame-interval |
5.0 |
Seconds between sampled frames (adaptive — grows if needed to respect max-frames) |
--output-dir, -o |
./output/videolens |
Where report.html + report.md + analysis.json are written |
--force |
off | Bypass cache and reprocess |
--json |
off | Skip terminal summary printout |
--verbose, -v |
off | Verbose logging |
# Turn a YouTube video into a detailed written report
videolens analyze "https://www.youtube.com/watch?v=..." --mode general \
--prompt "Write a thorough report with the important ideas, examples, caveats, and conclusions."
# Convert a tutorial into a guide
videolens analyze "https://www.youtube.com/watch?v=..." --mode tutorial \
--prompt "Create an ordered guide with prerequisites, exact steps, warnings, and verification checks."
# Extract only the information worth remembering
videolens analyze "https://www.youtube.com/watch?v=..." \
--prompt "Extract the key ideas, facts, examples, and conclusions without repetition or filler."| Mode | Use it for |
|---|---|
general |
Broad review: what's happening, what's notable, what's worth knowing |
bug |
Bug recordings → repro steps, severity hint, ticket-ready summary |
meeting |
Calls/standups/briefings → decisions, objections, commitments, follow-ups (uses diarized transcription) |
ux |
Session replays → user intent, friction, abandoned flows, and product fixes |
tutorial |
How-to videos → ordered steps, prerequisites, commands, warnings, and a checklist |
product_demo |
Product demos → feature inventory, positioning, proof, gaps, and opportunities |
content |
Video content → hook, pacing, clarity, proof, editing opportunities, and CTA |
privacy |
Share-ready review → possible credentials, personal data, internal URLs, and redaction plan |
production_recipe |
Reference videos → how the video itself was made: script spine, shot inventory, edit rhythm, likely tools, asset checklist, and recreation recipe |
Each mode is a small prompt-fragment file under src/videolens/analysis/modes/. Adding a new mode is ~30 lines.
Defaults are in src/videolens/config.py:
| Stage | Model |
|---|---|
| Transcription (general/bug) | gpt-4o-mini-transcribe |
| Transcription (meeting) | gpt-4o-transcribe-diarize |
| Frame describe + OCR (per frame) | gpt-5.4-mini |
| Final analysis synthesis | gpt-5.5 |
Swap them by editing the Models dataclass.
Rough order-of-magnitude:
- A 30-second video with 5 frames → well under $0.05
- A 5-minute video with 20 frames → typically under $0.20
- A 30-minute meeting with 40 frames → typically $0.50–$1.50
Per-prompt analysis cache means asking a second question of an already-extracted video costs only the synthesis call (cents).
These services store user-session events, not video — yt-dlp can't extract them. VideoLens ships an optional browser-capture fallback that opens the replay URL in headless Chromium, starts playback, and records the viewport in real time:
uv sync --extra capture
uv run playwright install chromiumAfter installation, paste any PostHog/Hotjar/etc. share URL and VideoLens routes it through Chromium automatically. Tune the recording window via --capture-duration <seconds> on the CLI (default 60s).
Self-host only for now. Real-time capture takes as long as the replay itself, which conflicts with the request lifecycle of the managed-PaaS deployment at app.videolens.io. The hosted instance falls back to the existing "screen-record manually and upload" workflow for replay services.
VideoLens ships a Model Context Protocol server so AI agents can analyse videos as a first-class tool.
uv sync --extra mcpAvailable tools:
| Tool | What it does |
|---|---|
analyze_video |
Run the full pipeline against a source + prompt + mode |
ask_video |
Follow-up question against a previously analysed video |
get_timeline |
Fetch the cached timeline (segments with visual/OCR/transcript) |
get_transcript |
Fetch the cached transcript |
get_frames |
Fetch frame summaries |
list_cached |
List all videos with cached extraction artifacts |
Add it to Claude Code:
claude mcp add videolens -- videolens-mcpOr wire it into Cursor / Windsurf / any MCP-aware host the same way as any stdio MCP server.
The VideoLens extension runs the whole pipeline inside the browser as a side-panel Chrome extension — no Python, no ffmpeg, no server. It analyzes the video on the current tab (YouTube captions + canvas frame sampling) or a local file (in-browser audio decode → transcription), using your own OpenAI key, which never leaves your device.
- Free and open source — unlimited analyses and no VideoLens account
- Four reader-focused report styles plus 6 specialized modes and follow-up Q&A
- Professional, self-contained HTML reports with print-ready PDF output
- Markdown/JSON export and copyable report text
- Source lives in
extension/(MIT, like everything here)
cd extension
npm install
npm run build # → extension/dist, load it unpacked via chrome://extensionsOr download the latest extension package, unzip it, open chrome://extensions, enable Developer mode, and choose Load unpacked. Version 0.2.0 is the first Chrome Web Store review package.
See extension/PUBLISHING.md for the release
runbook and extension/STORE_LISTING.md for the
Chrome Web Store listing. Known limits: DRM-protected players (Netflix etc.)
can't be captured; live streams aren't supported; non-YouTube in-page videos
are analyzed frames-only (no audio transcription).
- Robustness: soft-fail transcription so frame-only videos still produce a report; clearer error surfaces in the UI
- More sources: webpage-rendered embedded video (Playwright), session-replay JSON exports (PostHog / Clarity / Hotjar / FullStory / LogRocket / OpenReplay), Zoom/Meet/Teams recordings
- Scene-change frame selection: smarter than the adaptive interval used in v0.1
- Embeddings + semantic search over processed timelines
Issues and PRs welcome. The architecture is modular by design — most additions land in their own file under resolvers/, processors/, analysis/modes/, or outputs/ without touching the rest.
MIT — see LICENSE.