High-level PyTorch framework with Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks system, and minimal boilerplate. Scales from laptop to supercomputer with same code. Use when you want clean training loops with built-in best practices.

PyTorch Lightning High Level Training Framework Quick start PyTorch Lightning organizes PyTorch code to eliminate boilerplate while maintaining flexibility. Installation : Convert PyTorch to Lightning (3 steps): That's it! Trainer handles: GPU/TPU/CPU switching Distributed training (DDP, FSDP, DeepSpeed) Mixed precision (FP16, BF16) Gradient accumulation Checkpointing Logging Progress bars Common workflows Workflow 1: From PyTorch to Lightning Original PyTorch code : Lightning version : Benefits : 40+ lines → 15 lines, no device management, automatic distributed Workflow 2: Validation and testing Automatic features : Validation runs every epoch by default Metrics logged to TensorBoard Best model checkpointing based on val loss Workflow 3: Distributed training (DDP) Launch : No changes needed : Automatic data distribution Gradient synchronization Multi node support (just set num nodes=2 ) Workflow 4: Callbacks for monitoring Result : Auto saves best 3 models Stops early if no improvement for 5 epochs Logs learning rate to TensorBoard Workflow 5: Learning rate scheduling When to use vs alternatives Use PyTorch Lightning when : Want clean, organized code Need production ready training loops Switching between single GPU, multi GPU, TPU Want built in callbacks and logging Team collaboration (standardized structure) Key advantages : Organized : Separates research code from engineering Automatic : DDP, FSDP, DeepSpeed with 1 line Callbacks : Modular training extensions Reproducible : Less boilerplate = fewer bugs Tested : 1M+ downloads/month, battle tested Use alternatives instead : Accelerate : Minimal changes to existing code, more flexibility Ray Train : Multi node orchestration, hyperparameter tuning Raw PyTorch : Maximum control, learning purposes Keras : TensorFlow ecosystem Common issues Issue: Loss not decreasing Check data and model setup: Issue: Out of memory Reduce batch size or use gradient accumulation: Issue: Validation not running Ensure you pass val loader: Issue: DDP spawns multiple processes unexpectedly Lightning auto detects GPUs. Explicitly set devices: Advanced topics Callbacks : See [references/callbacks.md](references/callbacks.md) for EarlyStopping, ModelCheckpoint, custom callbacks, and callback hooks. Distributed strategies : See [references/distributed.md](references/distributed.md) for DDP, FSDP, DeepSpeed ZeRO integration, multi node setup. Hyperparameter tuning : See [references/hyperparameter tuning.md](references/hyperparameter tuning.md) for integration with Optuna, Ray Tune, and WandB sweeps. Hardware requirements CPU : Works (good for debugging) Single GPU : Works Multi GPU : DDP (default), FSDP, or DeepSpeed Multi node : DDP, FSDP, DeepSpeed TPU : Supported (8 cores) Apple MPS : Supported Precision options : FP32 (default) FP16 (V100, older GPUs) BF16 (A100/H100, recommended) FP8 (H100) Resources Docs: https://lightning.ai/docs/pytorch/stable/ GitHub: https://github.com/Lightning AI/pytorch lightning ⭐ 29,000+ Version: 2.5.5+ Examples: https://github.com/Lightning AI/pytorch lightning/tree/master/examples Discord: https://discord.gg/lightning ai Used by: Kaggle winners, research labs, production teams

pytorch-lightning