Use to run AutoMagicCalib on local MP4s, RTSP, or the bundled sample dataset, and to deploy vss-auto-calibration when needed. Do not use for non-AMC calibration or runtime analytics.

Purpose Run AutoMagicCalib end to end on local files, RTSP streams, or the bundled sample dataset and (when needed) deploy the AMC microservice. Instructions Follow the routing tables and step by step workflows below. Each section that ends in workflow , quick start , or flow is intended to be executed top to bottom. Detailed reference material lives in references/ ; load only the reference needed for the selected input mode. Examples Worked end to end examples are kept under evals/ (each .json manifest contains a runnable scenario) and inline in the per workflow curl blocks below. Run a Tier 3 evaluation with nv base validate <this skill dir agent eval to replay them. Limitations Requires the matching VSS profile / microservice to be deployed and reachable from the caller. NGC hosted models and NIMs may be subject to rate limits, GPU memory requirements, and license restrictions. Concurrency, GPU memory, and storage limits depend on the host hardware and the profile's compose file. Troubleshooting Error : REST call returns connection refused. Cause : target microservice not running. Solution : probe /docs or /health ; redeploy via vss deploy profile or the matching vss deploy skill. Error : HTTP 401/403 from NGC pulls. Cause : missing/expired NGC CLI API KEY . Solution : docker login nvcr.io and re export the key before retrying. Error : container OOM or model fails to load. Cause : insufficient GPU memory for the selected profile. Solution : switch to a smaller variant or free GPUs via docker compose down . VSS Generate Video Calibration Run AutoMagicCalib over one of three input sources and drive the calibration through the microservice REST API. The input resolution work differs per source; everything from verify project onward is identical and lives in this file. Pick the right input mode reference and pair it with the [Shared Calibration Tail]( shared calibration tail) below. Shared helper references are loaded only when needed: Read [ references/common steps.md ](references/common steps.md) when a mode reference needs the shared create project , video upload, or handoff snippets. Read [ references/calibration tail.md ](references/calibration tail.md) when you need the reusable Python implementation of the verify → calibrate → poll → results tail. Input Routing Match the user's request to a mode, then load that mode's reference for input collection, mode specific API calls, and the full Python script. User says / has Mode Reference "launch AMC" / "deploy auto calibration" / "set up auto magic calib" / "start AMC microservice" deploy [ references/deploy auto calibration service.md ](references/deploy auto calibration service.md) "calibrate my videos" / "calibrate from video files" / local cam .mp4 files videos [ references/videos.md ](references/videos.md) "calibrate RTSP streams" / "calibrate from live cameras" / live RTSP URLs rtsp [ references/rtsp.md ](references/rtsp.md) "test sample dataset" / "verify AMC install" / "launch and test" sample dataset [ references/sample dataset.md ](references/sample dataset.md) Disambiguation rule: if the user is asking to launch / deploy / set up AMC (no calibration verb) → deploy . If they provide RTSP URLs → rtsp . If they mention local files / a videos directory → videos . If they ask to verify install or test the bundled sample → sample dataset . Combined intents (e.g. "launch AMC and calibrate my videos") → walk deploy first, then the calibration mode. When ambiguous, ask via AskUserQuestion . Prerequisites (shared across calibration modes) AMC microservice + UI running. If not, walk [ references/deploy auto calibration service.md ](references/deploy auto calibration service.md) first. Microservice reachable at http://<HOST IP :${VSS AUTO CALIBRATION PORT: 8010}/v1/ready → {"code":0,...} . Projects directory writable by the container user. If you didn't just deploy (so Step 5 of the deploy reference hasn't run), confirm the write test in [ references/deploy auto calibration service.md § Step 5](references/deploy auto calibration service.md step 5 confirm the projects directory is writable) — otherwise the first create project returns [Errno 13] Permission denied . Python 3 with requests installed (each input mode reference includes a self healing venv fallback for direct runs). Mode specific prerequisites (VIOS for rtsp , sample zip for sample dataset ) live in the respective references. Shared Calibration Tail The verify → calibrate → poll → results sequence is identical regardless of input mode. After the mode specific reference has uploaded videos / ingested RTSP clips / uploaded the bundled sample, run this tail. Use [ references/calibration tail.md ](references/calibration tail.md) for the shared Python snippet. Step A — Verify Project Response: {"project state": "READY"} — must be READY before calibrating. If not READY, re check that videos + alignment + layout are present (either via API or via UI manual alignment). Step B — Start Calibration Confirm the plan before calibrating. Whether the settings file and detector were auto detected or asked, present a short summary and confirm via AskUserQuestion before the POST /calibrate . The resolved values are the defaults, so confirming is one click — but the user can switch the detector or skip an auto detected settings file. Summarize: Detector — resnet or transformer (the value to be sent). Calibration settings — the file being applied (path), or default parameters (with the option to tune them in the UI first — see below). Optional overrides — ground truth zip and focal lengths, if any. The sample dataset install check run uses a fixed resnet and can proceed without this confirmation. detector type is a separate /calibrate parameter — not consumed by /v1/config/<id . If the user provided a calibration settings file, parse it for "detector" / "detector type" and use that value. If the file doesn't specify one, the default ( resnet ) is the value shown in the confirmation above — the user can switch it there before calibrating. If there's no settings file at all, ask the user via AskUserQuestion : resnet — default, fast. transformer — slower, better under heavy occlusion. UI Step 3 (Parameters) does NOT cover detector choice; never assume the user picked one in the UI. Also when there's no settings file, ask whether to tune the calibration parameters first ( AskUserQuestion ): Proceed with the default parameters — well suited to typical warehouse scenes; recommended unless the user has specific tuning in mind. Adjust parameters in the UI first — open the project, go to Step 3: Parameters, change values, and click Save; then continue. Wait for the user's choice — and, if they choose to tune, for them to confirm they've Saved — before calling /calibrate . Step C — Poll for Completion Poll every 10 s. project info.project state : State Meaning RUNNING Calibration in progress COMPLETED Finished ERROR Failed — pull log via GET /v1/amc/calibrate/<id /log When calibration starts, surface the project ID, the UI URL ( http://<HOST IP :${VSS AUTO CALIBRATION UI PORT: 5000} ), and the log endpoint so the user can watch progress while the run proceeds. During RUNNING , emit a progress line at least once a minute with elapsed time so a long run doesn't look stalled. On ERROR , fetch and show the last lines of GET /v1/amc/calibrate/<id /log before stopping. Live logs can also be streamed via GET /v1/calibrate/<project id /log/<type /stream . Typical time: 10–60 min (your own videos), 10–30 min (bundled sample). Step D — Results Evaluation response includes Average L2 distance(m) and Average reprojection error 0(px) . Evaluation metrics are produced only when a ground truth GT.zip was uploaded — a missing evaluation statistics result is normal otherwise and is not the end of result reporting. After COMPLETED , always give the user a way to review the result for that exact project, regardless of whether metrics exist: UI — http://<HOST IP :${VSS AUTO CALIBRATION UI PORT: 5000} ; open the project, then the Results page to view the overlay. Overlay image on disk — ${VSS APPS DIR}/services/auto calibration/projects/project <id /output/multi view results/BA output/results ba scaled world/overlay img .png (single camera projects use output/single view results/cam 00/verification map overlay.png ). Project files — ${VSS APPS DIR}/services/auto calibration/projects/project <id / . Step E — VGGT Refinement After the AMC run completes, always check vggt state in project info. VGGT model staging is optional during setup and must not block the AMC result, but post AMC handling follows the state: If vggt state == "READY" and the user explicitly requested VGGT refinement or staged VGGT during this setup flow, run VGGT refinement without asking again. If vggt state == "READY" but VGGT was already staged before this request and the user has not asked for VGGT refined output, ask via AskUserQuestion whether to run refinement before starting it. If VGGT is not ready, skip refinement and mention that VGGT refinement is available after staging the model (see [ references/deploy auto calibration service.md ](references/deploy auto calibration service.md) Step 2). Settings File + Detector Pattern Optional across all three modes. When the user provides a JSON settings file (typically exported from UI Step 3 Download), POST it verbatim: The file replaces what the user would otherwise tune in UI Step 3 (rectification, bundle adjustment, evaluation knobs, detector, …). After a successful POST, also parse the file for "detector" / "detector type" — if it's "resnet" or "transformer" , use that value for the /calibrate call in Step B (detector is a separate API parameter, not consumed by /config ). Non 2xx is surfaced — do not silently fall back. Skip this call entirely if the user chose the UI fallback path. UI Fallback Pattern When alignment / layout files aren't on disk, direct the user to the appropriate AMC UI step: Settings missing → "Open UI project <project id , go to Step 3: Parameters , tune via the settings dialog (or accept defaults), click Save." Also : before the /calibrate call, ask the user via AskUserQuestion whether to use the resnet or transformer detector — Step 3 doesn't cover detector choice. Layout missing → "Open UI project <project id , go to Step 2: Video Configuration , upload layout.png only (do NOT re upload videos — they're already attached via API/RTSP), click Save." Alignment missing → "Open UI project <project id , go to Step 4: Alignment , either upload alignment data.json or mark correspondence points on the layout, click Save." Wait for user confirmation. For alignment/layout, verify on disk before continuing: Success Criteria project state == "COMPLETED" after polling. If manual alignment was used: ${VSS APPS DIR}/services/auto calibration/projects/project <id /manual adjustment/ contains alignment data.json + layout.png . If GT was uploaded: evaluation returns typical thresholds ( Average L2 distance(m) < 1.5, Average reprojection error 0(px) < 5 for your data; < 10 for the bundled sample). No ERROR state. Key Output Files Under ${VSS APPS DIR}/services/auto calibration/projects/project <project id / : Cross cutting Troubleshooting Mode specific issues live in each reference's own troubleshooting table. Issue Fix verify project state not READY Confirm videos uploaded/ingested and alignment + layout are present (either via API or via UI manual alignment). Mode specific upload steps in the reference. Ma

vss-generate-video-calibration