Watch a video (URL or local path). Downloads with yt-dlp, extracts auto-scaled frames with ffmpeg, pulls the transcript from captions (or Whisper API fallback), and hands the result to Claude so it can answer questions about what's in the video.

/watch You don't have a video input; this skill gives you one. A Python script gets captions first, optionally downloads the video, extracts frames as JPEGs (scene aware, or fast keyframes at efficient detail), gets a timestamped transcript (native captions first, then Whisper API as fallback), and prints frame paths. You then Read each frame path to see the images and combine them with the transcript to answer the user. Resolve SKILL DIR (do this before any command) Every python3 ... command below runs a bundled script under SKILL DIR/scripts/ . Set SKILL DIR to the absolute path of the directory containing THIS SKILL.md you just Read — your harness told you that path in the Read result. The scripts are always a direct sibling of this file ( SKILL DIR/scripts/watch.py ), in every install layout: Substitute that literal path for ${SKILL DIR} in every command. This works on every harness (Claude Code, Codex, Cursor, Gemini CLI, …) without relying on any harness specific environment variable. Guard once at the start of a run: Step 0 — Setup preflight (runs every /watch invocation, silent on success) Python interpreter: every python3 ... command in this skill is for macOS/Linux. On Windows , substitute python — the python3 command on Windows is the Microsoft Store stub and will not run the script. On the first /watch invocation in a session, use structured preflight so you can detect first run setup: Branch on two fields: can proceed: true and first run: false → setup is already done (the user may have deliberately skipped a Whisper key — that's allowed). Proceed to Step 1 without comment. first run: true → genuine first time setup. Do these in order: 1. If missing binaries is non empty, run the installer first (it auto installs on macOS / prints commands elsewhere — see below) and confirm the binaries land. Do not skip this and jump to preferences. 2. Run the installer once more if needed so it scaffolds ~/.config/watch/.env (it only writes the template when the file is absent, so let it create the file before you write any values into it). 3. Encourage a Whisper API key and ask the watch preference questions below, then write the selected values into ~/.config/watch/.env and set SETUP COMPLETE=true . can proceed: false and first run: false → setup was finished before but the environment regressed (e.g. missing binaries after an OS change). Run the installer to remediate, then proceed. Don't re ask preferences. A missing Whisper key is encouraged to fix, not required : on a genuine first run status will read needs key even when binaries are present — that's your cue to encourage a key, not a blocker. On follow up /watch calls in the same session, use the silent check: This is a <100ms lookup. Exit 0 means /watch can run — this includes a user who finished setup without a Whisper key (keyless is allowed). On exit 0 the script emits nothing — proceed to Step 1 without comment. Do NOT announce "setup is complete" to the user — they don't need a status message on every turn. The only acceptable user visible output from Step 0 is when remediation is required. On non zero exit, follow the table: Exit Meaning Action 2 Missing binaries ( ffmpeg / ffprobe / yt dlp ) Run installer 3 Genuine first run with no Whisper API key Run installer to scaffold .env , then encourage a key (the user may decline — proceed with no whisper ) 4 Both missing Run installer, then encourage a key Exit 3 only fires before the user has completed setup. Once SETUP COMPLETE=true is written, a keyless install returns exit 0 and is never nagged again. The installer is idempotent — safe to re run: On macOS with Homebrew, it auto installs ffmpeg and yt dlp . On Linux/Windows, it prints the exact install commands for the user to run. It scaffolds ~/.config/watch/.env with commented placeholders and default watch settings at 0600 perms. If an API key is still missing after install: use AskUserQuestion to ask the user whether they have a Groq API key (preferred — cheaper, faster) or an OpenAI key. Then write it into ~/.config/watch/.env — set the matching GROQ API KEY=... or OPENAI API KEY=... line. If they don't want to set up Whisper, proceed with no whisper and tell them videos without native captions will come back frames only. First run watch preference: after the installer has scaffolded ~/.config/watch/.env , use AskUserQuestion to ask one question: Default detail (one dial). Present these as AskUserQuestion options in this exact order — lightest to heaviest — and keep (recommended) on balanced even though it is not first (do not reorder to put the recommended option first): transcript — no frames at all, transcript only (skips video download when captions exist). efficient — fast keyframe pass (cap 50). balanced (recommended) — scene aware frames (cap 100, default). token burner — scene aware, uncapped (maximum fidelity; high token cost). Write the answer directly into ~/.config/watch/.env by setting the bare key on its own line — no trailing inline comment (a note after the value can break parsing): Use the user's selected value. If they skip the question, keep the recommended default. Once dependencies, the API key choice, and this preference are handled, write or update SETUP COMPLETE=true in the same file. Do not ask this preference question again when SETUP COMPLETE=true . Structured mode (optional): python3 "${SKILL DIR}/scripts/setup.py" json emits {status, can proceed, first run, setup complete, missing binaries, whisper backend, has api key, config file, watch detail, platform} where status is one of ready needs install needs key needs install and key . status describes the ideal state (a key is encouraged, so a keyless first run reads needs key ); can proceed is the operational gate (binaries present AND a key is set OR setup was already completed). Branch on can proceed / first run to decide whether to run; use status to decide what to encourage. Within a single session, you can skip Step 0 on follow up /watch calls — once check returned 0, nothing about the environment changes between turns. When to use User pastes a video URL (YouTube, Vimeo, X, TikTok, Twitch clip, most yt dlp supported sites) and asks about it. User points at a local video file ( .mp4 , .mov , .mkv , .webm , etc.) and asks about it. User types /watch <url or path [question] . Recommended limits Best accuracy: videos under 10 minutes. Frame coverage scales inversely with duration. Universal rate cap: 2 fps. The script never samples faster than 2 fps, even when a budget or fps would imply more. The frame ceiling is set by the detail mode ( WATCH DETAIL in ~/.config/watch/.env , or detail ), not a single global cap: transcript → no frames efficient → up to 50 (keyframes) balanced (default) → up to 100 (scene aware) token burner → uncapped (scene aware; a soft warning prints past 250 frames) max frames N overrides whichever cap the mode would otherwise use. Full video frame budget by duration. Token cost grows with frame count, so the script targets a budget by duration. This budget sets the fps and the uniform sampling fallback; scene aware selection can fill up to the detail cap above, whichever is lower: ≤30s → ~12 30 frames 30s 1min → ~40 frames 1 3min → ~60 frames 3 10min → ~80 frames \ 10min → up to the detail cap, sparsely spaced (warning printed) If the user hands you a long video, consider asking whether they want a specific section before burning tokens on a sparse scan. How to invoke Step 1 — parse the user input. Separate the video source (URL or path) from any question the user asked. Example: /watch https://youtu.be/abc what language is this in? → source = https://youtu.be/abc , question = what language is this in? . Step 2 — run the watch script. Pass the source verbatim. Do not shell escape it yourself beyond normal quoting: Optional flags: detail transcript efficient balanced token burner — fidelity/speed dial. transcript = no frames (transcript only, skips video download when captions exist); efficient = fast keyframes (cap 50); balanced = scene aware frames (cap 100); token burner = scene aware, uncapped. start T / end T — focus on a section. Accepts SS , MM:SS , or HH:MM:SS . When either is set, fps auto scales denser (see "Focusing on a section" below). timestamps T1,T2,… — grab a frame at each of these absolute timestamps ( SS , MM:SS , or HH:MM:SS ). Use this after reading the transcript to capture deictic moments the presenter flags ("look here", "as you can see", "notice this") that visual selection alone may miss. See "Transcript cue frames" below. max frames N — override the preset cap for tighter token budget (e.g. max frames 40 ) resolution W — change frame width in px (default 512; bump to 1024 only if the user needs to read on screen text) fps F — override auto fps (clamped to 2 fps max) out dir DIR — keep working files somewhere specific (default: an auto generated tmp dir) whisper groq openai — force a specific Whisper backend (default: prefer Groq if both keys exist) no whisper — disable the Whisper fallback entirely (frames only if no captions) no dedup — keep near duplicate frames. By default a frame delta pass drops frames that are visually near identical to the previous kept one (held slides, static screen recordings, paused video) so the frame budget goes to distinct content; the report's Frames line notes how many were dropped. Pass this only if the user needs every sampled frame (e.g. judging subtle frame to frame motion). Focusing on a section (higher frame rate) When the user asks about a specific moment — "what happens at the 2 minute mark?", "zoom into 0:45 to 1:00", "the first 10 seconds" — pass start and/or end . The script switches to focused mode budgets, which are denser than full video budgets (still capped at 2 fps, and still bounded by the detail mode cap — the counts below assume the default balanced cap of 100; efficient tops out at 50): ≤5s → 2 fps (up to 10 frames) 5 15s → 2 fps (up to 30 frames) 15 30s → ~2 fps (up to 60 frames) 30 60s → ~1.3 fps (up to 80 frames) 60 180s → ~0.6 fps (100 frames, capped) Focused mode is the right call for: Any moment/range the user names explicitly ("around 2:30", "the intro", "the last 30 seconds"). Any video longer than ~10 minutes where the user's question is about a specific part — running focused on the relevant section is far more useful than a sparse scan of the whole thing. Re runs after a full scan didn't have enough detail in some region. Transcript is auto filtered to the same range. Frame timestamps are absolute (real video timeline, not offset from start). Examples: Step 3 — Read every frame path the script lists. The Read tool renders JPEGs directly as images for you. Read all frames in a single message (parallel tool calls) so you see them together. The frames are in chronological order with a t=MM:SS timestamp so you can align them to the transcript. Step 4 — answer the user. You now have two streams of evidence: Frames — what's on screen at each timestamp Transcript — what's said at each timestamp. The report's header shows the source ( captions = yt dlp pulled native subs; whisper (groq) or whisper (openai) = transcribed by API). If the user asked a specific question, answer it directly citing timestamps. If they didn't ask anything, summarize what happens in the video — structure, key moments, notable visuals, spoken content. This holds for transcript detail too: even with no frames, produce a summary like the other modes