NVIDIA DeepStream SDK development with Python pyservicemaker API. Use when building video analytics pipelines, GStreamer-based video processing, TensorRT inference integration, object detection/tracking, or Kafka/message broker integration.

DeepStream Development Skill This skill requires access to all of the reference documents listed in the references/ directory below. Ensure they are available before executing the workflow. When this skill is active, ALWAYS read the relevant reference documents before generating code. Do NOT rely on memory the reference documents contain critical details about exact property names, correct API usage, and common pitfalls. SDK and Architecture Quick Reference DeepStream SDK Version Requirements GStreamer : 1.24.2 NVIDIA Driver : 590+ CUDA : 13.1 TensorRT : 10.14.1.48 Platforms : Ubuntu 24.04 (x86 64 and ARM64/Jetson) Typical Pipeline Flow Components in [brackets] are optional only add them when the user explicitly requests them. Stage Role Key Element(s) Required? Source Input from files, RTSP, cameras nvurisrcbin (preferred), nvmultiurisrcbin , filesrc Yes Stream Muxer Batches streams for inference nvstreammux Yes Inference TensorRT model execution nvinfer , nvinferserver Yes Tracker Multi object tracking across frames nvtracker Only if requested OSD Draws bounding boxes, labels, overlays nvosdbin Yes (for visualization) Renderer Display or save output nveglglessink , nv3dsink , filesink Yes Memory Model DeepStream uses NVIDIA Video Memory Manager (NVMM) for zero copy GPU buffer transfers. Caps strings use memory:NVMM to indicate GPU memory (e.g., video/x raw(memory:NVMM), format=NV12 ). Critical Rules 1. Only Add Requested Components : Do NOT add pipeline elements the user did not ask for. Tracker ( nvtracker ) : Only add when the user explicitly requests tracking or object IDs across frames Secondary GIEs : Only add when the user requests classification or attribute extraction Analytics ( nvdsanalytics ) : Only add when the user requests line crossing, ROI counting, etc. Message broker ( nvmsgbroker / nvmsgconv ) : Only add when the user requests Kafka/cloud messaging When in doubt, build the minimal working pipeline and let the user ask for additions 2. Default to nvurisrcbin for Sources : When the user says "camera", "stream", "video", or provides a file path: Always use nvurisrcbin it handles RTSP, HTTP, and local files ( file:// ) transparently Only use filesrc + qtdemux + parser when the user explicitly needs raw file source control For RTSP/live sources, also set live source=1 on nvstreammux and sync=0 on the sink Convert local paths to URI: "file://" + os.path.abspath(path) 3. Metadata Iteration : Use .frame items and .object items (returns iterators, NOT lists) NEVER use len() on these iterate to count Iterator can only be consumed once 4. Request Pad Syntax : Use "sink %u" template, NEVER literal pad names 5. Platform Detection for Sinks : For WSL2 Ubuntu 24 Docker, this default selection must be overridden. WSL2 + Ubuntu 24 Docker : If /proc/version contains microsoft or wsl and /etc/os release has VERSION ID="24.04" , the generated app must never create a display branch or display sink ( nveglglessink , nv3dsink , etc.), even if the prompt asks for display. Do not rely on a no display flag for this case. Generate encoded MP4 output only ( nvv4l2h264enc h264parse mp4mux / qtmux filesink ) and make the default run path write the annotated video file. In the generated README.md , explicitly explain that WSL2 Ubuntu 24 Docker is MP4 output only because display sinks are disabled by a known issue. If the user explicitly requested display, add an inline code comment and README note explaining: Display requested but disabled due to WSL2 Ubuntu 24 Docker limitation — MP4 output generated instead. Non WSL targets : Do not add WSL specific behavior or WSL limitation text to generated apps or READMEs. Use the normal platform display sink selection above. 6. Buffer Cloning : Always clone buffers for async processing 7. Queue Types : queue.Queue → Use with threading.Thread multiprocessing.Queue → Use with multiprocessing.Process Using wrong type causes silent data loss! 8. nvinfer Config Format : YAML: Use property: section (NOT model: ), key: value with space after colon INI: Use [property] section, key=value with equals sign Section MUST be named property 9. nvmsgbroker is a SINK : Cannot have downstream elements use tee to split pipeline 10. ALL Sinks Need async=0 for Tee Splits or Dynamic Sources : CRITICAL for state transitions Symptom if missing : Pipeline stays in PAUSED state, no video displays. 11. Built in Probe Attachment : measure fps probe can only be attached to processing elements (e.g., nvinfer , nvosdbin ), NOT to sink elements. Attaching to a sink raises RuntimeError: Probe failure . 12. Dynamic ONNX Models Require infer dims : When the ONNX model has dynamic input shapes (e.g., exported with dynamic=True in Ultralytics YOLO, or with dynamic batch/height/width axes), you MUST add infer dims=C;H;W to the nvinfer config. Without it, TensorRT sees 1 for dynamic dimensions and fails with setDimensions: Error Code 3 . Common values: YOLO models (640 input): infer dims=3;640;640 Models with 416 input: infer dims=3;416;416 Models with 1280 input: infer dims=3;1280;1280 13. Ultralytics YOLO Output Format Depends on Model Generation — newer models (v10+/v26+) output post NMS results; older models (v8/v11) output raw pre NMS tensors. The custom parser and cluster mode must match the actual output: Model generation Output tensor shape Fields cluster mode v8 / v11 [batch, 84, 8400] [features(4+80), anchors] — raw cx/cy/w/h + class scores, no NMS 2 (NMS) v10 / v26+ [batch, 300, 6] [max det, (x1,y1,x2,y2,conf,cls)] — already post NMS, pixel coords 4 (none) How to identify at runtime : log inferDims.d[0] and inferDims.d[1] inside the custom parser. d={84, 8400} → pre NMS (v8/v11 style) d={300, 6} → post NMS (v10/v26+ style) Symptom of mismatch : If cluster mode: 2 is used with a post NMS [N, 6] output, bounding boxes appear shifted by 45° or 135° from the actual objects (DeepStream's NMS incorrectly re processes already final coordinates). If you see tilted or rotated boxes, also check the OBB / rotation angle note in references/nvinfer config.md : for non OBB models, value initialize NvDsInferObjectDetectionInfo with obj{} and keep rotation angle = 0 ; plain NvDsInferObjectDetectionInfo obj; leaves fields uninitialized. 14. Virtual Environment Must Include pyservicemaker : pyservicemaker is installed system wide but is NOT accessible from a standard Python virtual environment. When a task requires a venv (e.g., for model download/conversion pip dependencies), always install pyservicemaker and pyyaml inside the venv ; do not rewrite pyservicemaker pipeline code into non pyservicemaker code to work around a missing import. The venv setup in generated code and README must always include: Symptom if missing : ModuleNotFoundError: No module named 'pyservicemaker' when running the app inside the venv. Key Paths Models: /opt/nvidia/deepstream/deepstream/samples/models/ Primary Detector: /opt/nvidia/deepstream/deepstream/samples/models/Primary Detector/resnet18 trafficcamnet pruned.onnx Tracker lib: /opt/nvidia/deepstream/deepstream/lib/libnvds nvmultiobjecttracker.so Kafka lib: /opt/nvidia/deepstream/deepstream/lib/libnvds kafka proto.so Sample configs: /opt/nvidia/deepstream/deepstream/samples/configs/deepstream app/ Reference Documents IMPORTANT : Always read these documents for complete details. Do NOT generate code from memory. Document Use When [references/gstreamer plugins.md](references/gstreamer plugins.md) Looking up plugin properties, ALL properties listed [references/service maker api.md](references/service maker api.md) Using Pipeline/Flow API, metadata access, probes, EventMessageUserMetadata [references/use cases pipelines.md](references/use cases pipelines.md) Building pipelines: simple playback, multi inference, cascaded GIE [references/streaming sources.md](references/streaming sources.md) Ingesting local files, HTTP MP4, HLS, MPEG DASH, or RTSP sources with nvurisrcbin [references/kafka messaging.md](references/kafka messaging.md) Kafka/message broker setup, nvmsgconv/nvmsgbroker config, msg2p newapi [references/best practices.md](references/best practices.md) Design patterns, common pitfalls, anti patterns [references/buffer apis.md](references/buffer apis.md) BufferProvider/Feeder (injection), BufferRetriever/Receiver (extraction) [references/media extractor advanced.md](references/media extractor advanced.md) MediaExtractor, MediaChunk, FrameSampler [references/utilities config.md](references/utilities config.md) PerfMonitor, EngineFileMonitor, SourceConfig, SensorInfo, SmartRecordConfig [references/nvinfer config.md](references/nvinfer config.md) nvinfer config file format, ALL parameters [references/tracker config.md](references/tracker config.md) nvtracker config, NvDCF/IOU/DeepSORT/NvSORT [references/troubleshooting.md](references/troubleshooting.md) Error messages and solutions [references/rest api dynamic.md](references/rest api dynamic.md) REST API, dynamic source add/remove, nvmultiurisrcbin [references/metamux config.md](references/metamux config.md) nvdsmetamux config, parallel multi model inference, metadata merging, source ID filtering [references/docker containers.md](references/docker containers.md) Docker images, Dockerfile examples, pyservicemaker install, container run commands [references/nvds msgapi adapter.md](references/nvds msgapi adapter.md) Building custom protocol adapters: nvds msgapi Quick Error Reference Error Solution iterator has no len() Iterate to count, don't use len() pad template not found Use "sink %u" not "sink 0" Queue data loss Use multiprocessing.Queue with Process Config parse failed Use property: not model: in YAML is classifier deprecation warning Use network type: 1 instead of is classifier: 1 for classifiers; omit both for detectors min boxes unknown key warning Use minBoxes (camelCase) in class attrs sections, not min boxes Secondary GIE inactive Set process mode: 2 , check operate on gie id Tee/dynamic source stuck PAUSED Set async: 0 on ALL sink elements WSL2 Ubuntu 24 display sink requested Do not use display sinks due to a known bug; write MP4 with filesink and document the WSL limitation in README RTSP no data/reconnecting Test URL with ffplay, check credentials RuntimeError: Probe failure measure fps probe cannot attach to sink elements; use nvinfer or nvosdbin instead setDimensions negative dims / engine build failed Add infer dims=C;H;W for dynamic ONNX models (e.g., infer dims=3;640;640 ) No module named 'pyservicemaker' in venv pip install /opt/nvidia/deepstream/deepstream/service maker/python/pyservicemaker .whl pyyaml inside the venv AttributeError: object has no attribute 'obj label' Use obj meta.label not obj meta.obj label in pyservicemaker (C API name differs from Python binding) <! Signing refresh marker.

deepstream-dev