Reduce LLM size and accelerate inference using pruning techniques like Wanda and SparseGPT. Use when compressing models without retraining, achieving 50% sparsity with minimal accuracy loss, or enabling faster inference on hardware accelerators. Covers unstructured pruning, structured pruning, N:M s

Model Pruning: Compressing LLMs When to Use This Skill Use Model Pruning when you need to: Reduce model size by 40 60% with <1% accuracy loss Accelerate inference using hardware friendly sparsity (2 4× speedup) Deploy on constrained hardware (mobile, edge devices) Compress without retraining using one shot methods Enable efficient serving with reduced memory footprint Key Techniques : Wanda (weights × activations), SparseGPT (second order), structured pruning, N:M sparsity Papers : Wanda ICLR 2024 (arXiv 2306.11695), SparseGPT (arXiv 2301.00774) Installation Quick Start Wanda Pruning (One Shot, No Retraining) Source : ICLR 2024 (arXiv 2306.11695) SparseGPT (Second Order Pruning) Source : arXiv 2301.00774 N:M Structured Pruning (Hardware Accelerator) Core Concepts 1. Pruning Criteria Magnitude Pruning (baseline): Wanda (weights × activations): SparseGPT (second order): 2. Structured vs Unstructured Unstructured (fine grained): Prune individual weights Higher quality (better accuracy) No hardware speedup (irregular sparsity) Structured (coarse grained): Prune entire neurons, heads, or layers Lower quality (more accuracy loss) Hardware speedup (regular sparsity) Semi structured (N:M) : Best of both worlds 50% sparsity (2:4) → 2× speedup on NVIDIA GPUs Minimal accuracy loss 3. Sparsity Patterns Pruning Strategies Strategy 1: Gradual Magnitude Pruning Strategy 2: Layer wise Pruning Strategy 3: Iterative Pruning + Fine tuning Production Deployment Complete Pruning Pipeline Evaluation Best Practices 1. Sparsity Selection 2. Method Selection 3. Avoid Common Pitfalls Performance Comparison Pruning methods at 50% sparsity (LLaMA 7B): Method Accuracy Loss Speed Memory Retraining Needed Magnitude 2.5% 1.0× 50% No Wanda 0.8% 1.0× 50% No SparseGPT 0.4% 1.0× 50% No N:M (2:4) 1.0% 2.0× 50% No Structured 3.0% 2.0× 50% No Source : Wanda paper (ICLR 2024), SparseGPT paper Resources Wanda Paper (ICLR 2024) : https://arxiv.org/abs/2306.11695 Wanda GitHub : https://github.com/locuslab/wanda SparseGPT Paper : https://arxiv.org/abs/2301.00774 SparseGPT GitHub : https://github.com/IST DASLab/sparsegpt NVIDIA Sparse Tensor Cores : https://developer.nvidia.com/blog/accelerating inference with sparsity using ampere and tensorrt/

model-pruning