Analyse video content by extracting frames with ffmpeg and using AI vision to generate timestamped step-by-step summaries. Use when user provides a video file and wants to understand its visual content — screen recordings, tutorials, presentations, footage, or animations. Triggers on "analyse this v

FFmpeg Video Analysis Extract frames from video files with ffmpeg. Delegate frame reading to sub agents to preserve the main context window. Synthesise a structured timestamped summary from text only sub agent reports. Architecture: Context Efficient Sub Agent Pipeline Problem : Reading dozens of images into the main conversation context consumes most of the context window, leaving little room for synthesis and follow up. Solution : A 3 phase pipeline: Images only ever exist inside sub agent contexts. The main agent only reads lightweight text files. This cuts context usage by ~90%. 1. Prerequisites If either is missing, show platform specific install instructions and STOP: macOS : brew install ffmpeg Ubuntu/Debian : sudo apt install ffmpeg Windows : choco install ffmpeg or winget install ffmpeg 2. Setup Temp Directory 3. Extract Video Metadata Extract and report: duration, resolution (width x height), fps, codec, file size, whether audio is present. If no video stream is found, report "audio only file" and STOP. If file size 2GB, warn the user and suggest analysing a time range with ss START to END . 4. Extract Frames Choose strategy based on duration: Duration Strategy Command 0 60s 1 frame every 2s ffmpeg hide banner y i INPUT vf "fps=1/2,scale='min(1280,iw)': 2" q:v 5 DIR/frame %04d.jpg 1 10min Scene detection (threshold 0.3) ffmpeg hide banner y i INPUT vf "select='gt(scene,0.3)',scale='min(1280,iw)': 2" vsync vfr q:v 5 DIR/scene %04d.jpg 10 30min Keyframe extraction ffmpeg hide banner y skip frame nokey i INPUT vf "scale='min(1280,iw)': 2" vsync vfr q:v 5 DIR/key %04d.jpg 30min+ Thumbnail filter ffmpeg hide banner y i INPUT vf "thumbnail=SEGMENT FRAMES,scale='min(1280,iw)': 2" vsync vfr q:v 5 DIR/thumb %04d.jpg For thumbnail filter, calculate SEGMENT FRAMES = total frames / 60 to cap output at ~60 frames. Fallbacks: Scene detection yields 0 frames → retry with interval at 1 frame/5s More than 100 frames extracted → subsample evenly to 80 Frame extraction fails → try the next simpler strategy (scene → interval, keyframe → interval) Time range analysis: When user specifies a range, prepend ss START to END before i . Higher detail mode: If requested, double the fps rate and lower scene threshold to 0.2. After extraction, list all frame files and calculate each frame's timestamp from its sequence number and the extraction rate. 5. Delegate Frame Analysis to Sub Agents This is the critical context saving step. Do NOT read frame images in the main conversation. Instead, split frames into batches and delegate each batch to a sub agent. 5a. Prepare Batch Manifest Split the extracted frame file list into batches of 8 10 frames each. For each batch, record: Batch number (1, 2, 3, ...) Frame file paths (absolute) Frame timestamps (calculated from sequence number) Output file path: TMPDIR/batch N analysis.md 5b. Spawn Sub Agents For each batch, spawn a sub agent with the prompt below. Launch all batches in parallel where the tool supports it — they are fully independent. Sub Agent Prompt Template Use this prompt verbatim, substituting the placeholders: How to Spawn Use whatever sub agent, background task, or independent agent mechanism your tool provides. The requirements are simple — each sub agent needs to: 1. Read image files (the frame JPEGs) 2. Write a text file (the batch analysis markdown) Launch all batches in parallel if your tool supports it — they are fully independent with no shared state. If your tool has no sub agent mechanism , fall back to reading frames directly in the main context but limit to 20 frames maximum and warn the user about context usage. 5c. Collect Results After all sub agents complete, read the text analysis files. These are lightweight markdown — no images enter the main context. Read each batch N analysis.md file in order . These contain only text descriptions — the context cost is minimal compared to reading the original images. 6. Synthesise Output Using only the text from the batch analysis files, perform synthesis in the main context: 1. Merge all frame descriptions into a single chronological timeline 2. Group frames into natural segments (same scene, slide, or screen) 3. Detect the dominant content type across all batches 4. Identify 3 7 key moments 5. Extract all quoted text, prompts, or commands the user typed 6. Write a 2 5 sentence narrative summary Format the output as: 7. Cleanup Remove the temp directory after output is complete: Skip cleanup if the user asks to keep frames. Advanced Options Time range : "Analyse 2:00 to 5:00 of video.mp4" → use ss 120 to 300 Higher detail : "Analyse in high detail" → double frame rate, lower scene threshold to 0.2 Focus area : "Focus on the code shown" → prioritise text/code extraction in sub agent prompts Sprite sheet : For a visual overview, generate a contact sheet: Error Handling ffmpeg not found → install instructions per platform, STOP No video stream → report audio only, STOP Scene detection yields 0 frames → fallback to interval Too many frames ( 100) → subsample to 80 Large files ( 2GB) → warn, suggest time range Sub agent fails or times out → read that batch's frames directly as fallback, warn about context usage Frame read failure in sub agent → skip frame, note gap in batch analysis file

ffmpeg-analyse-video