PyTorch deep learning patterns and best practices for building robust, efficient, and reproducible training pipelines, model architectures, and data loading. Use when writing or reviewing PyTorch training loops, model architectures, or data loading, or when a run will not reproduce.

PyTorch Development Patterns Idiomatic PyTorch patterns and best practices for building robust, efficient, and reproducible deep learning applications. When to Activate Writing new PyTorch models or training scripts Reviewing deep learning code Debugging training loops or data pipelines Optimizing GPU memory usage or training speed Setting up reproducible experiments Core Principles 1. Device Agnostic Code Always write code that works on both CPU and GPU without hardcoding devices. 2. Reproducibility First Set all random seeds for reproducible results. 3. Explicit Shape Management Always document and verify tensor shapes. Model Architecture Patterns Clean nn.Module Structure Proper Weight Initialization Training Loop Patterns Standard Training Loop Validation Loop Data Pipeline Patterns Custom Dataset Efficient DataLoader Configuration Custom Collate for Variable Length Data Checkpointing Patterns Save and Load Checkpoints Performance Optimization Mixed Precision Training Gradient Checkpointing for Large Models torch.compile for Speed Quick Reference: PyTorch Idioms Idiom Description model.train() / model.eval() Always set mode before train/eval torch.no grad() Disable gradients for inference optimizer.zero grad(set to none=True) More efficient gradient clearing .to(device) Device agnostic tensor/model placement torch.amp.autocast Mixed precision for 2x speed pin memory=True Faster CPU→GPU data transfer torch.compile JIT compilation for speed (2.0+) weights only=True Secure model loading torch.manual seed Reproducible experiments gradient checkpointing Trade compute for memory Anti Patterns to Avoid Remember : PyTorch code should be device agnostic, reproducible, and memory conscious. When in doubt, profile with torch.profiler and check GPU memory with torch.cuda.memory summary() .

pytorch-patterns