Prepare datasets and configure LoRA training for character consistency. Covers FLUX (AI-Toolkit, SimpleTuner, FluxGym) and SDXL (Kohya_ss) training with step-by-step guidance. Use when training custom character LoRAs.

ComfyUI LoRA Training Guide the user through dataset preparation, training configuration, and evaluation for character LoRAs. When to Train vs Zero Shot Scenario Recommendation Need absolute consistency across many images Train LoRA Building a character series or ongoing project Train LoRA Quick one off generation Use zero shot (InstantID/PuLID) Limited references (1 5 images) Use zero shot Testing concepts Use zero shot first, train if committing Training Pipeline Dataset Preparation Image Requirements Aspect Minimum Optimal Maximum Count 10 15 20 30 50+ Resolution 512x512 1024x1024 Format PNG/high JPEG PNG Content Diversity Checklist [ ] Multiple angles (front, 3/4, profile, back) [ ] Various expressions (neutral, smile, serious, laugh, etc.) [ ] Different lighting conditions (studio, natural, dramatic) [ ] Varied backgrounds (or transparent/solid) [ ] Multiple outfits/contexts [ ] Some close ups, some medium shots [ ] If from 3D renders: include style variations (see below) Preprocessing 3D Renders Problem : Training directly on 3D renders bakes in the "3D" aesthetic. Solution : Generate style variations first: 1. Run each render through img2img with varied style prompts 2. Mix: 60% style variations, 40% original renders 3. This teaches identity, not style Style prompts for variation: Captioning Rules Trigger word : ALWAYS use a unique token as first word. Good: sage character , ohwx sage , sks person Bad: woman , redhead , character (too generic) Caption structure: DO NOT describe face features (let the model learn them): Bad: "woman with green eyes, freckles, auburn hair, defined cheekbones" Good: "sage character, woman, indoor portrait, wearing blue sweater" DO describe everything else : clothing, pose, background, lighting, expression. Folder Structure Folder naming: 10 sage character = each image repeated 10x per epoch. Training Configurations FLUX LoRA (AI Toolkit) Recommended FLUX training notes: Converges 2 3x faster than SDXL 1000 2000 steps usually sufficient Watch for overfitting (quality plateaus early) 24GB VRAM for standard, 9GB with NF4 quantization (SimpleTuner) SDXL LoRA (Kohya ss) Proven Step calculation: Low VRAM Training (FluxGym / SimpleTuner) For 12 16GB VRAM: Evaluation Protocol Test Each Checkpoint Use identical prompts across all checkpoints: Quality Indicators Good training: Character recognizable from trigger word alone Responds to different prompts/contexts Doesn't always produce same pose/expression Prompt 4 does NOT produce the character Overfitting signs: Same exact pose/expression regardless of prompt Training backgrounds appearing in outputs Ignores clothing/setting prompts Prompt 4 produces the character (too strong) Best Epoch Selection If using sample every: 250 with 1500 steps: Checkpoint 250: Usually underfit Checkpoint 500 750: Often sweet spot for FLUX Checkpoint 1000 1500: May be overfitting Compare visually and select the checkpoint with best identity + prompt flexibility balance. Post Training Integration 1. Copy best checkpoint to {ComfyUI}/models/loras/ 2. Update character profile: 3. Test in full workflow: LoRA (0.7 0.9) + PuLID/IP Adapter (0.5 0.7) 4. Record successful settings in character's generation history Combining LoRA with Zero Shot Methods Best practice: LoRA as base identity, zero shot for enhancement. Lower weights on both prevents conflict while reinforcing identity. Troubleshooting Issue Solution LoRA not activating Check trigger word spelling, ensure loaded before KSampler Identity drift at angles Add more angle variety to dataset, reduce network dim Overfitting Reduce epochs, increase dataset, lower network dim Style contamination Better caption diversity, don't describe style in captions Poor quality/artifacts Check training images for compression, reduce LR Reference references/lora training.md Full parameter reference references/models.md Training tool download links Character profiles in projects/ for trigger words and reference images

comfyui-lora-training