nanochat LLM Training Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection. nanochat is Karpathy's minimal, hackable harness for training LLMs end to end on a single GPU node. It covers tokenization, pretraining, SFT finetuning, RL, evaluation (DCLM CORE score), inference with KV cache, and a ChatGPT like web UI. A single complexity dial ( depth ) auto configures all other hyperparameters (width, heads, LR, training horizon, weight decay) for compute optimal training. You can reproduce GPT 2 capability (~$43,000 in 2019) for ~$48 on an 8×H100 node (~2 hours). Installation nanochat uses uv for dependency management: Key Commands Full GPT 2 Speedrun (8×H100 node, ~2–3 hours, ~$48) Pretraining (distributed) Pretraining (single GPU) Quick Research Iteration (~5 min, GPT 1 scale) CPU / Apple Silicon (tiny model, ~minutes) Serve Chat UI CLI Chat Scaling Laws / Miniseries The Depth Dial The single most important parameter. Everything else is derived automatically: depth Approximate model scale Notes 6–8 Tiny (toy) CPU/MPS feasible 12 GPT 1 size ~5 min on 8×H100, great for research iteration 16 Medium ~15 min on 8×H100 24–26 GPT 2 size ~2 hrs on 8×H100, ~$48 Precision / dtype Configuration nanochat uses explicit dtype management via COMPUTE DTYPE in nanochat/common.py . No torch.amp.autocast . Hardware Default Override CUDA SM 80+ (A100, H100) bfloat16 NANOCHAT DTYPE=float32 CUDA SM < 80 (V100, T4) float32 NANOCHAT DTYPE=float16 CPU / MPS float32 — How it works: Weights stored in fp32 (optimizer precision), custom Linear casts to COMPUTE DTYPE in forward pass, embeddings stored directly in COMPUTE DTYPE to save memory. Key Python Modules Real Code Examples Load and Run Inference on a Trained Model Custom Training Script with Depth Dial Adjust Device Batch Size for Lower VRAM Monitoring Key Metrics in wandb Synthetic Data for SFT Personality Common Patterns Research Iteration Loop FP8 Training (H100 only, for speedrun) Evaluate CORE Score Only Serve on Lambda / Remote Machine Troubleshooting OOM / Out of VRAM Single GPU is 8× Slower This is expected. Omit torchrun and use python m scripts.base train directly. Gradient accumulation kicks in automatically to maintain equivalent total batch size. Running on Non CUDA Hardware float16 Gradient Underflow V100 / T4 (SM < 80) — No bf16 Chat UI Not Accessible Resources DeepWiki Q&A : https://deepwiki.com/karpathy/nanochat Discussions : https://github.com/karpathy/nanochat/discussions Discord : nanochat channel on Karpathy's Discord Leaderboard docs : dev/LEADERBOARD.md Beating GPT 2 guide : https://github.com/karpathy/nanochat/discussions/481 Miniseries v1 : https://github.com/karpathy/nanochat/discussions/420 Adding abilities guide : https://github.com/karpathy/nanochat/discussions/164

nanochat-llm-training