Control LLM output with regex and grammars, guarantee valid JSON/XML/code generation, enforce structured formats, and build multi-step workflows with Guidance - Microsoft Research's constrained generation framework

Guidance: Constrained LLM Generation When to Use This Skill Use Guidance when you need to: Control LLM output syntax with regex or grammars Guarantee valid JSON/XML/code generation Reduce latency vs traditional prompting approaches Enforce structured formats (dates, emails, IDs, etc.) Build multi step workflows with Pythonic control flow Prevent invalid outputs through grammatical constraints GitHub Stars : 18,000+ From : Microsoft Research Installation Quick Start Basic Example: Structured Generation With Anthropic Claude Core Concepts 1. Context Managers Guidance uses Pythonic context managers for chat style interactions. Benefits: Natural chat flow Clear role separation Easy to read and maintain 2. Constrained Generation Guidance ensures outputs match specified patterns using regex or grammars. Regex Constraints How it works: Regex converted to grammar at token level Invalid tokens filtered during generation Model can only produce matching outputs Selection Constraints 3. Token Healing Guidance automatically "heals" token boundaries between prompt and generation. Problem: Tokenization creates unnatural boundaries. Solution: Guidance backs up one token and regenerates. Benefits: Natural text boundaries No awkward spacing issues Better model performance (sees natural token sequences) 4. Grammar Based Generation Define complex structures using context free grammars. Use cases: Complex structured outputs Nested data structures Programming language syntax Domain specific languages 5. Guidance Functions Create reusable generation patterns with the @guidance decorator. Stateful Functions: Backend Configuration Anthropic Claude OpenAI Local Models (Transformers) Local Models (llama.cpp) Common Patterns Pattern 1: JSON Generation Pattern 2: Classification Pattern 3: Multi Step Reasoning Pattern 4: ReAct Agent Pattern 5: Data Extraction Best Practices 1. Use Regex for Format Validation 2. Use select() for Fixed Categories 3. Leverage Token Healing 4. Use stop Sequences 5. Create Reusable Functions 6. Balance Constraints Comparison to Alternatives Feature Guidance Instructor Outlines LMQL Regex Constraints ✅ Yes ❌ No ✅ Yes ✅ Yes Grammar Support ✅ CFG ❌ No ✅ CFG ✅ CFG Pydantic Validation ❌ No ✅ Yes ✅ Yes ❌ No Token Healing ✅ Yes ❌ No ✅ Yes ❌ No Local Models ✅ Yes ⚠️ Limited ✅ Yes ✅ Yes API Models ✅ Yes ✅ Yes ⚠️ Limited ✅ Yes Pythonic Syntax ✅ Yes ✅ Yes ✅ Yes ❌ SQL like Learning Curve Low Low Medium High When to choose Guidance: Need regex/grammar constraints Want token healing Building complex workflows with control flow Using local models (Transformers, llama.cpp) Prefer Pythonic syntax When to choose alternatives: Instructor: Need Pydantic validation with automatic retrying Outlines: Need JSON schema validation LMQL: Prefer declarative query syntax Performance Characteristics Latency Reduction: 30 50% faster than traditional prompting for constrained outputs Token healing reduces unnecessary regeneration Grammar constraints prevent invalid token generation Memory Usage: Minimal overhead vs unconstrained generation Grammar compilation cached after first use Efficient token filtering at inference time Token Efficiency: Prevents wasted tokens on invalid outputs No need for retry loops Direct path to valid outputs Resources Documentation : https://guidance.readthedocs.io GitHub : https://github.com/guidance ai/guidance (18k+ stars) Notebooks : https://github.com/guidance ai/guidance/tree/main/notebooks Discord : Community support available See Also references/constraints.md Comprehensive regex and grammar patterns references/backends.md Backend specific configuration references/examples.md Production ready examples