Use when the user asks for 4k vfx or a task matching the examples below. Turn a plain video clip into a cinematic 4K AI-VFX shot. Give Claude a video and the change you want; it reads EVERY frame via local contact sheets and understands the audio, writes a Seedance-faithful prompt that locks your fa

4k vfx Maps a video to video VFX move onto the Pika MCP's Seedance 4K reference to video . The user hands you a clip + the change they want; you read EVERY frame (locally extracted with ffmpeg and tiled into contact sheets) and understand the audio (speech + music + SFX + ambience), author a Seedance faithful prompt that locks the original (face, gestures, camera move) and time codes the change to the right beat, then re render the same shot in 4K with the VFX baked in. One run produces one 4K clip for one requested change. How it works — why reading EVERY frame (and the audio) is the mechanism (do not skip the reading pass) Seedance treats the reference video as a motion/style anchor, NOT a pixel locked base — it does not copy the input frame for frame, it re generates the shot guided by your prompt. So fidelity comes from the PROMPT : the more exhaustively the prompt describes the original (subject identity, exact gestures, camera move, framing, lighting, palette, wardrobe) and the audio (dialogue + lip sync, music/SFX beats), the more the 4K output reads as the same shot with the change layered on. Reading EVERY frame — not a sample — is how you build that exhaustive description: you locally extract all frames with ffmpeg, tile them 25 to a contact sheet (5×5), and analyze every sheet so the prompt is built from the entire timeline rather than a handful of stills. Skip the read (or read only a sample) and the output drifts — a different face, a different camera move, a different room, a missed motion beat. The reading pass — all frames + the audio — IS the skill. Prerequisites pika MCP available, and local ffmpeg in the run environment (used to extract every frame and build the contact sheets). Two inputs from the user: (1) a video clip (a local file or a public URL) and (2) the VFX change they want made to it. Tools used: upload asset , probe media , analyze media , transcribe audio , estimate cost , generate reference video , task status (frame extraction + tiling is done locally with ffmpeg, not an MCP tool). Stage 0 — Gather inputs (settle this first) You need exactly two things before generating: 1. The video — the original clip to transform. If they gave a local file, you will upload it (Step 1); if they gave a public URL, keep it. 2. The change / VFX — what should be different in the output (e.g. "turn the plaza into an open desert on a finger snap", "make it snow", "set the room on fire behind me", "morph my jacket into glowing armor"). If either is missing, ask for it before doing anything else — do not invent a change, and do not proceed on a clip you cannot reach. Once both are in hand, say in one line what you're about to do ("Reading every frame of your clip to build the desert on snap 4K prompt…") and run Steps 1–5 without further check ins — but stop at the Step 6 agreement gate before generating : the 4K render is the expensive, irreversible step, and it only fires after the user approves the prompt and cost. Step 1 — Upload the clip ( upload asset ) Real video must be uploaded — inline base64 only works for tiny assets (<~3MB), so any actual clip goes through the presigned upload path. 1. Call upload asset with filename , mime type ( video/mp4 for mp4), and size bytes (the file's byte size). 2. PUT the raw bytes to the returned presigned url . 3. Keep the returned public url → state.clip url . If the user already gave a public URL , skip this step and set state.clip url to it. Step 2 — Probe ( probe media ) Call probe media on state.clip url to read duration, fps, dimensions, and aspect ratio . Use these to: (a) sanity check the all frame extraction in Step 3 (duration × fps ≈ total frame count ≈ 25 × number of sheets), and (b) choose the output aspect ratio and duration here, matched to the source — the Step 6 gate quotes this choice to the user and Step 7 fires with it. Step 3 — Extract EVERY frame locally and tile into contact sheets (ffmpeg) Read the whole video, not a sample. Use local ffmpeg to extract every frame at the source rate (no drops, no exceptions), then tile them into legible contact sheets at 25 frames per sheet (5×5 grid) so each analyze media call covers 25 consecutive frames at once. Work in a relative working directory (do not use an absolute system temp path): Then build the 25 per sheet contact sheets (5×5), scaling each frame down so the sheet is legible and stamping the frame index onto each tile: drawtext=text='%{n}' stamps the frame index on each tile so the analysis can reference exact frames; tile=5x5 groups 25 consecutive frames per sheet. If drawtext is unavailable in the local ffmpeg build, fall back to plain scale=480: 1,tile=5x5 . INPUT is the local clip (or a local copy of the URL). Confirm the sheet count lines up with the probe in Step 2 — every frame must land on a sheet, no exceptions. Step 4 — Read EVERY sheet + understand the audio ( upload asset , analyze media , transcribe audio ) This is the fidelity step, and it has two mandatory halves: read all the frames and understand the audio . Neither is optional. 4a — Read all the frames (every sheet) For each contact sheet from Step 3 (cover ALL of them — every frame is on a sheet, no exceptions): upload it with upload asset (the sheets are images, so upload to get a URL), then call analyze media on that sheet URL with a query that reads the full progression across those 25 frames — pulling out everything the prompt must lock: Subject / identity — who/what is in frame; face, hair, build, distinguishing features. Exact motion & gestures — what the subject does, beat by beat, referencing the stamped frame indices (e.g. "raises right hand around frame 0048, snaps fingers around frame 0072"). Camera move — static / pan / push in / handheld / orbit, and its timing. Framing — shot size (close/medium/wide), subject position, headroom. Lighting — direction, hardness, color temperature, time of day. Palette — dominant colors, mood. Wardrobe / props / setting — clothing, objects, background. Carry the read across all sheets so the entire timeline is understood start to end, not just a sampled middle. 4b — Understand the audio (mandatory, not just speech) Always read the audio — it is required, not optional. Two passes: transcribe audio for speech + timestamps (the words and when they're said), so the prompt can preserve dialogue and lip sync beats. analyze media on the audio (or the video URL) to understand the NON speech audio too — music, sound effects, ambience, tone, and rhythm/beats. Fold the audio understanding into the prompt where it matters — preserve dialogue + lip sync if anyone speaks, and time the VFX change to an audio beat / sound cue when the change should land on the music or a sound (e.g. snap, hit, downbeat) rather than a bare timestamp. Assemble the all frame read and the audio read into a single faithful description of the original — that description is the raw material for Step 5. Step 5 — Write the Seedance prompt (the authoring step, NEVER skip it) Compose ONE prompt = a faithful, exhaustive description of the original (the locked elements) + the user's requested change, time coded + a reference to the input clip via the @Video1 token. Be thorough: every detail you read in Step 4 that you omit is a detail Seedance is free to change. Template — fill the [SLOTS] from your Step 4 read: Time code the change so Seedance knows when it happens relative to the locked motion (e.g. "at 0–3s the plaza is unchanged as the subject raises their hand; at 3s on the finger snap the plaza dissolves into an open desert, sand and heat haze replacing the buildings while the subject, camera move, and framing stay identical"). The stronger and more specific the time coded change clause, the less Seedance ignores it. Step 6 — Agreement gate ( estimate cost ) — MANDATORY before generating The 4K render is the expensive step, and once fired it can't be un spent. Never call generate reference video without explicit user approval in this session. Present, in one message: 1. The full Step 5 prompt — verbatim, so the user can catch a wrong detail (face, gesture, timing) before it costs money. 2. The generation recipe — provider seedance , model standard , resolution 4k , plus the aspect ratio and duration you chose from the Step 2 probe. 3. The estimated cost — call estimate cost for the Step 7 generate reference video call and quote the result. If the estimate errors, say the cost is unknown and flag that 4K standard is the top price tier — do not silently skip the number. Then wait . Three outcomes: Approve → proceed to Step 7 unchanged. Revise — the user corrects a detail or the change clause → update the Step 5 prompt (re reading specific sheets if the correction demands it) and re present the gate. Abort / no answer → do not generate. Never treat silence as approval. Only skip this gate if the user explicitly pre authorized the spend in this session ("just run it, don't ask", a standing instruction to render without confirmation). Having provided the clip and the change is NOT pre authorization — that's just Stage 0 input. Step 7 — Generate (Seedance 4K reference to video) ( generate reference video ) Call generate reference video with the exact 4K recipe: provider : seedance seedance model : standard — REQUIRED for 4k (fast/mini cap at 720p). resolution : 4k reference videos : [state.clip url] — the original footage as the motion/style anchor (Seedance accepts up to 3 videos). reference images : optional — style/creature/identity anchors (e.g. a generated reference image), only if you have one. prompt : the Step 5 prompt (uses the @Video1 token to reference the input clip; no max length, so be exhaustive). aspect ratio : auto (or match the source from Step 2). duration : matched to the source, OR set auto duration : true . Do NOT pass negative prompt (not supported on seedance — the call is rejected) and do NOT pass any fps param (it doesn't exist). Step 8 — Deliver If the call returns inline, you have the 4K video URL. If it returns { task id, status: running/background } , poll task status(task id) in a tight loop until completed / failed / cancelled . On completion, return the 4K video URL to the user. Note: the output CDN may be egress blocked for in session download — so deliver the URL itself rather than trying to fetch the bytes locally. Failure modes symptom cause → fix Clip too long / too large Upload still works for large files; if Seedance rejects on duration, trim the source to the segment that matters, or set auto duration: true and match a shorter duration . No face / subject found in the read A sheet may not show the subject clearly — re run analyze media on the specific sheet(s) covering the frames where the subject is visible (use the stamped frame indices), or rebuild the sheets without downscaling so detail is preserved. Do NOT drop to a sampled extraction — every frame stays on a sheet. ffmpeg missing / drawtext unavailable The run environment needs local ffmpeg ; if the build lacks the drawtext filter, fall back to plain scale=480: 1,tile=5x5 (sheets without stamped indices — still read every frame). Identity / shot drift (different face, room, or camera move in the output) The prompt description was not exhaustive enough. Go back to Step 4–5 and add the missing locked details (more on face, gestures, camera move, lighting) from the contact sheet read — Seedance only preserves what the prompt names. Re render = new spend: re pass the Step 6 gate with the revised prompt before firing again. 4k rejected / output downgraded Ensure seedance model: standard (fast/mini cap at 720p) and res

4k-vfx