Train or fine-tune language and vision models using TRL (Transformer Reinforcement Learning) or Unsloth with Hugging Face Jobs infrastructure. Covers SFT, DPO, GRPO and reward modeling training methods, plus GGUF conversion for local deployment. Includes guidance on the TRL Jobs package, UV scripts

TRL Training on Hugging Face Jobs Overview Train language models using TRL (Transformer Reinforcement Learning) on fully managed Hugging Face infrastructure. No local GPU setup required—models train on cloud GPUs and results are automatically saved to the Hugging Face Hub. TRL provides multiple training methods: SFT (Supervised Fine Tuning) Standard instruction tuning DPO (Direct Preference Optimization) Alignment from preference data GRPO (Group Relative Policy Optimization) Online RL training Reward Modeling Train reward models for RLHF For detailed TRL method documentation: See also: references/training methods.md for method overviews and selection guidance When to Use This Skill Use this skill when users want to: Fine tune language models on cloud GPUs without local infrastructure Train with TRL methods (SFT, DPO, GRPO, etc.) Run training jobs on Hugging Face Jobs infrastructure Convert trained models to GGUF for local deployment (Ollama, LM Studio, llama.cpp) Ensure trained models are permanently saved to the Hub Use modern workflows with optimized defaults When to Use Unsloth Use Unsloth ( references/unsloth.md ) instead of standard TRL when: Limited GPU memory Unsloth uses ~60% less VRAM Speed matters Unsloth is ~2x faster Training large models ( 13B) memory efficiency is critical Training Vision Language Models (VLMs) Unsloth has FastVisionModel support See references/unsloth.md for complete Unsloth documentation and scripts/unsloth sft example.py for a production ready training script. Key Directives When assisting with training jobs: 1. ALWAYS use hf jobs() MCP tool Submit jobs using hf jobs("uv", {...}) , NOT bash trl jobs commands. The script parameter accepts Python code directly. Do NOT save to local files unless the user explicitly requests it. Pass the script content as a string to hf jobs() . If user asks to "train a model", "fine tune", or similar requests, you MUST create the training script AND submit the job immediately using hf jobs() . 2. Always include Trackio Every training script should include Trackio for real time monitoring. Use example scripts in scripts/ as templates. 3. Provide job details after submission After submitting, provide job ID, monitoring URL, estimated time, and note that the user can request status checks later. 4. Use example scripts as templates Reference scripts/train sft example.py , scripts/train dpo example.py , etc. as starting points. Local Script Execution Repository scripts use PEP 723 inline dependencies. Run them with uv run : Prerequisites Checklist Before starting any training job, verify: ✅ Account & Authentication Hugging Face Account with [Pro](https://hf.co/pro), [Team](https://hf.co/enterprise), or [Enterprise](https://hf.co/enterprise) plan (Jobs require paid plan) Authenticated login: Check with hf whoami() HF TOKEN for Hub Push ⚠️ CRITICAL Training environment is ephemeral, must push to Hub or ALL training results are lost Token must have write permissions MUST pass secrets={"HF TOKEN": "$HF TOKEN"} in job config to make token available (the $HF TOKEN syntax references your actual token value) ✅ Dataset Requirements Dataset must exist on Hub or be loadable via datasets.load dataset() Format must match training method (SFT: "messages"/text/prompt completion; DPO: chosen/rejected; GRPO: prompt only) ALWAYS validate unknown datasets before GPU training to prevent format failures (see Dataset Validation section below) Size appropriate for hardware (Demo: 50 100 examples on t4 small; Production: 1K 10K+ on a10g large/a100 large) ⚠️ Critical Settings Timeout must exceed expected training time Default 30min is TOO SHORT for most training. Minimum recommended: 1 2 hours. Job fails and loses all progress if timeout is exceeded. Hub push must be enabled Config: push to hub=True , hub model id="username/model name" ; Job: secrets={"HF TOKEN": "$HF TOKEN"} Asynchronous Job Guidelines ⚠️ IMPORTANT: Training jobs run asynchronously and can take hours Action Required When user requests training: 1. Create the training script with Trackio included (use scripts/train sft example.py as template) 2. Submit immediately using hf jobs() MCP tool with script content inline don't save to file unless user requests 3. Report submission with job ID, monitoring URL, and estimated time 4. Wait for user to request status checks don't poll automatically Ground Rules Jobs run in background Submission returns immediately; training continues independently Initial logs delayed Can take 30 60 seconds for logs to appear User checks status Wait for user to request status updates Avoid polling Check logs only on user request; provide monitoring links instead After Submission Provide to user: ✅ Job ID and monitoring URL ✅ Expected completion time ✅ Trackio dashboard URL ✅ Note that user can request status checks later Example Response: Quick Start: Three Approaches 💡 Tip for Demos: For quick demos on smaller GPUs (t4 small), omit eval dataset and eval strategy to save ~40% memory. You'll still see training loss and learning progress. Sequence Length Configuration TRL config classes use max length (not max seq length ) to control tokenized sequence length: Default behavior: max length=1024 (truncates from right). This works well for most training. When to override: Longer context : Set higher (e.g., max length=2048 ) Memory constraints : Set lower (e.g., max length=512 ) Vision models : Set max length=None (prevents cutting image tokens) Usually you don't need to set this parameter at all the examples below use the sensible default. Approach 1: UV Scripts (Recommended—Default Choice) UV scripts use PEP 723 inline dependencies for clean, self contained training. This is the primary approach for Claude Code. Benefits: Direct MCP tool usage, clean code, dependencies declared inline (PEP 723), no file saving required, full control When to use: Default choice for all training tasks in Claude Code, custom training logic, any scenario requiring hf jobs() Working with Scripts ⚠️ Important: The script parameter accepts either inline code (as shown above) OR a URL. Local file paths do NOT work. Why local paths don't work: Jobs run in isolated Docker containers without access to your local filesystem. Scripts must be: Inline code (recommended for custom training) Publicly accessible URLs Private repo URLs (with HF TOKEN) Common mistakes: Correct approaches: To use local scripts: Upload to HF Hub first: Approach 2: TRL Maintained Scripts (Official Examples) TRL provides battle tested scripts for all methods. Can be run from URLs: Benefits: No code to write, maintained by TRL team, production tested When to use: Standard TRL training, quick experiments, don't need custom code Available: Scripts are available from https://github.com/huggingface/trl/tree/main/examples/scripts Finding More UV Scripts on Hub The uv scripts organization provides ready to use UV scripts stored as datasets on Hugging Face Hub: Popular collections: ocr, classification, synthetic data, vllm, dataset creation Approach 3: HF Jobs CLI (Direct Terminal Commands) When the hf jobs() MCP tool is unavailable, use the hf jobs CLI directly. ⚠️ CRITICAL: CLI Syntax Rules Key syntax rules: 1. Command order is hf jobs uv run (NOT hf jobs run uv ) 2. All flags ( flavor , timeout , secrets ) must come BEFORE the script URL 3. Use secrets (plural), not secret 4. Script URL must be the last positional argument Complete CLI example: Check job status via CLI: Approach 4: TRL Jobs Package (Simplified Training) The trl jobs package provides optimized defaults and one liner training. Benefits: Pre configured settings, automatic Trackio integration, automatic Hub push, one line commands When to use: User working in terminal directly (not Claude Code context), quick local experimentation Repository: https://github.com/huggingface/trl jobs ⚠️ In Claude Code context, prefer using hf jobs() MCP tool (Approach 1) when available. Hardware Selection Model Size Recommended Hardware Cost (approx/hr) Use Case <1B params t4 small ~$0.75 Demos, quick tests only without eval steps 1 3B params t4 medium , l4x1 ~$1.50 2.50 Development 3 7B params a10g small , a10g large ~$3.50 5.00 Production training 7 13B params a10g large , a100 large ~$5 10 Large models (use LoRA) 13B+ params a100 large , a10g largex2 ~$10 20 Very large (use LoRA) GPU Flavors: cpu basic/upgrade/performance/xl, t4 small/medium, l4x1/x4, a10g small/large/largex2/largex4, a100 large, h100/h100x8 Guidelines: Use LoRA/PEFT for models 7B to reduce memory Multi GPU automatically handled by TRL/Accelerate Start with smaller hardware for testing See: references/hardware guide.md for detailed specifications Critical: Saving Results to Hub ⚠️ EPHEMERAL ENVIRONMENT—MUST PUSH TO HUB The Jobs environment is temporary. All files are deleted when the job ends. If the model isn't pushed to Hub, ALL TRAINING IS LOST . Required Configuration In training script/config: In job submission: Verification Checklist Before submitting: [ ] push to hub=True set in config [ ] hub model id includes username/repo name [ ] secrets parameter includes HF TOKEN [ ] User has write access to target repo See: references/hub saving.md for detailed troubleshooting Timeout Management ⚠️ DEFAULT: 30 MINUTES—TOO SHORT FOR TRAINING Setting Timeouts Timeout Guidelines Scenario Recommended Notes Quick demo (50 100 examples) 10 30 min Verify setup Development training 1 2 hours Small datasets Production (3 7B model) 4 6 hours Full datasets Large model with LoRA 3 6 hours Depends on dataset Always add 20 30% buffer for model/dataset loading, checkpoint saving, Hub push operations, and network delays. On timeout: Job killed immediately, all unsaved progress lost, must restart from beginning Choose a Base Model (Model Selection) Identify models to train based on task type or benchmark results. Use scripts/hf benchmarks.py to identify top performing models for specific tasks. This helps the user select a model as the base for training, whilst keeping size and hardware constraints in mind. Example choosing an OCR base model Cost Estimation Offer to estimate cost when planning jobs with known parameters. Use scripts/estimate cost.py : Output includes estimated time, cost, recommended timeout (with buffer), and optimization suggestions. When to offer: User planning a job, asks about cost/time, choosing hardware, job will run 1 hour or cost $5 Example Training Scripts Production ready templates with all best practices: Load these scripts for correctly: scripts/train sft example.py Complete SFT training with Trackio, LoRA, checkpoints scripts/train dpo example.py DPO training for preference learning scripts/train grpo example.py GRPO training for online RL These scripts demonstrate proper Hub saving, Trackio integration, checkpoint management, and optimized parameters. Pass their content inline to hf jobs() or use as templates for custom scripts. Monitoring and Tracking Trackio provides real time metrics visualization. See references/trackio guide.md for complete setup guide. Key points: Add trackio to dependencies Configure trainer with report to="trackio" and run name="meaningful name" Trackio Configuration Defaults Use sensible defaults unless user

huggingface-llm-trainer