Evaluate Agent Skill design quality against official specifications and best practices. Use when reviewing, auditing, or improving SKILL.md files and skill packages. Provides multi-dimensional scoring and actionable improvement suggestions.

Skill Judge Evaluate Agent Skills against official specifications and patterns derived from 17+ official examples. Core Philosophy What is a Skill? A Skill is NOT a tutorial. A Skill is a knowledge externalization mechanism . Traditional AI knowledge is locked in model parameters. To teach new capabilities: Skills change this: This is the paradigm shift from "training AI" to "educating AI" — like a hot swappable LoRA adapter that requires no training. You edit a Markdown file in natural language, and the model's behavior changes. The Core Formula Good Skill = Expert only Knowledge − What Claude Already Knows A Skill's value is measured by its knowledge delta — the gap between what it provides and what the model already knows. Expert only knowledge : Decision trees, trade offs, edge cases, anti patterns, domain specific thinking frameworks — things that take years of experience to accumulate What Claude already knows : Basic concepts, standard library usage, common programming patterns, general best practices When a Skill explains "what is PDF" or "how to write a for loop", it's compressing knowledge Claude already has. This is token waste — context window is a public resource shared with system prompts, conversation history, other Skills, and user requests. Tool vs Skill Concept Essence Function Example Tool What model CAN do Execute actions bash, read file, write file, WebSearch Skill What model KNOWS how to do Guide decisions PDF processing, MCP building, frontend design Tools define capability boundaries — without bash tool, model can't execute commands. Skills inject knowledge — without frontend design Skill, model produces generic UI. The equation : Same Claude model, different Skills loaded, becomes different experts. Three Types of Knowledge in Skills When evaluating, categorize each section: Type Definition Treatment Expert Claude genuinely doesn't know this Must keep — this is the Skill's value Activation Claude knows but may not think of Keep if brief — serves as reminder Redundant Claude definitely knows this Should delete — wastes tokens The art of Skill design is maximizing Expert content, using Activation sparingly, and eliminating Redundant ruthlessly. Evaluation Dimensions (120 points total) D1: Knowledge Delta (20 points) — THE CORE DIMENSION The most important dimension. Does the Skill add genuine expert knowledge? Score Criteria 0 5 Explains basics Claude knows (what is X, how to write code, standard library tutorials) 6 10 Mixed: some expert knowledge diluted by obvious content 11 15 Mostly expert knowledge with minimal redundancy 16 20 Pure knowledge delta — every paragraph earns its tokens Red flags (instant score ≤5): "What is [basic concept]" sections Step by step tutorials for standard operations Explaining how to use common libraries Generic best practices ("write clean code", "handle errors") Definitions of industry standard terms Green flags (indicators of high knowledge delta): Decision trees for non obvious choices ("when X fails, try Y because Z") Trade offs only an expert would know ("A is faster but B handles edge case C") Edge cases from real world experience "NEVER do X because [non obvious reason]" Domain specific thinking frameworks Evaluation questions : 1. For each section, ask: "Does Claude already know this?" 2. If explaining something, ask: "Is this explaining TO Claude or FOR Claude?" 3. Count paragraphs that are Expert vs Activation vs Redundant D2: Mindset + Appropriate Procedures (15 points) Does the Skill transfer expert thinking patterns along with necessary domain specific procedures ? The difference between experts and novices isn't "knowing how to operate" — it's "how to think about the problem." But thinking patterns alone aren't enough when Claude lacks domain specific procedural knowledge. Key distinction : Type Example Value Thinking patterns "Before designing, ask: What makes this memorable?" High — shapes decision making Domain specific procedures "OOXML workflow: unpack → edit XML → validate → pack" High — Claude may not know this Generic procedures "Step 1: Open file, Step 2: Edit, Step 3: Save" Low — Claude already knows Score Criteria 0 3 Only generic procedures Claude already knows 4 7 Has domain procedures but lacks thinking frameworks 8 11 Good balance: thinking patterns + domain specific workflows 12 15 Expert level: shapes thinking AND provides procedures Claude wouldn't know What counts as valuable procedures : Workflows Claude hasn't been trained on (new tools, proprietary systems) Correct ordering that's non obvious (e.g., "validate BEFORE packing, not after") Critical steps that are easy to miss (e.g., "MUST recalculate formulas after editing") Domain specific sequences (e.g., MCP server's 4 phase development process) What counts as redundant procedures : Generic file operations (open, read, write, save) Standard programming patterns (loops, conditionals, error handling) Common library usage that's well documented Expert thinking patterns look like : Valuable domain procedures look like : Redundant generic procedures look like : The test : 1. Does it tell Claude WHAT to think about? (thinking patterns) 2. Does it tell Claude HOW to do things it wouldn't know? (domain procedures) A good Skill provides both when needed. D3: Anti Pattern Quality (15 points) Does the Skill have effective NEVER lists? Why this matters : Half of expert knowledge is knowing what NOT to do. A senior designer sees purple gradient on white background and instinctively cringes — "too AI generated." This intuition for "what absolutely not to do" comes from stepping on countless landmines. Claude hasn't stepped on these landmines. It doesn't know Inter font is overused, doesn't know purple gradients are the signature of AI generated content. Good Skills must explicitly state these "absolute don'ts." Score Criteria 0 3 No anti patterns mentioned 4 7 Generic warnings ("avoid errors", "be careful", "consider edge cases") 8 11 Specific NEVER list with some reasoning 12 15 Expert grade anti patterns with WHY — things only experience teaches Expert anti patterns (specific + reason): Weak anti patterns (vague, no reasoning): The test : Would an expert read the anti pattern list and say "yes, I learned this the hard way"? Or would they say "this is obvious to everyone"? D4: Specification Compliance — Especially Description (15 points) Does the Skill follow official format requirements? Special focus on description quality. Score Criteria 0 5 Missing frontmatter or invalid format 6 10 Has frontmatter but description is vague or incomplete 11 13 Valid frontmatter, description has WHAT but weak on WHEN 14 15 Perfect: comprehensive description with WHAT, WHEN, and trigger keywords Frontmatter requirements : name : lowercase, alphanumeric + hyphens only, ≤64 characters description : THE MOST CRITICAL FIELD — determines if skill gets used at all Why description is THE MOST IMPORTANT field : The brutal truth : A Skill with perfect content but poor description is useless — it will never be activated. The description is the only chance to tell the Agent "use me in these situations." Description must answer THREE questions : 1. WHAT : What does this Skill do? (functionality) 2. WHEN : In what situations should it be used? (trigger scenarios) 3. KEYWORDS : What terms should trigger this Skill? (searchable terms) Excellent description (all three elements): Analysis: WHAT: creation, editing, analysis, tracked changes, comments WHEN: "When Claude needs to work with... for: (1)... (2)... (3)..." KEYWORDS: .docx files, tracked changes, professional documents Poor description (missing elements): Problems: WHAT: vague ("文档相关功能" — what specifically?) WHEN: missing (when should Agent use this?) KEYWORDS: missing (no ".docx", no specific scenarios) Another poor example : This is useless — Agent has no idea when to activate it. Description quality checklist : [ ] Lists specific capabilities (not just "helps with X") [ ] Includes explicit trigger scenarios ("Use when...", "When user asks for...") [ ] Contains searchable keywords (file extensions, domain terms, action verbs) [ ] Specific enough that Agent knows EXACTLY when to use it [ ] Includes scenarios where this skill MUST be used (not just "can be used") D5: Progressive Disclosure (15 points) Does the Skill implement proper content layering? Skill loading has three layers: Score Criteria 0 5 Everything dumped in SKILL.md ( 500 lines, no structure) 6 10 Has references but unclear when to load them 11 13 Good layering with MANDATORY triggers present 14 15 Perfect: decision trees + explicit triggers + "Do NOT Load" guidance For Skills WITH references directory , check Loading Trigger Quality: Trigger Quality Characteristics Poor References listed at end, no loading guidance Mediocre Some triggers but not embedded in workflow Good MANDATORY triggers in workflow steps Excellent Scenario detection + conditional triggers + "Do NOT Load" The loading problem : Good loading trigger (embedded in workflow): Bad loading trigger (just listed): For simple Skills (no references, <100 lines): Score based on conciseness and self containment. D6: Freedom Calibration (15 points) Is the level of specificity appropriate for the task's fragility? Different tasks need different levels of constraint. This is about matching freedom to fragility. Score Criteria 0 5 Severely mismatched (rigid scripts for creative tasks, vague for fragile ops) 6 10 Partially appropriate, some mismatches 11 13 Good calibration for most scenarios 14 15 Perfect freedom calibration throughout The freedom spectrum : Task Type Should Have Why Example Skill Creative/Design High freedom Multiple valid approaches, differentiation is value frontend design Code review Medium freedom Principles exist but judgment required code review File format operations Low freedom One wrong byte corrupts file, consistency critical docx, xlsx, pdf High freedom (text based instructions): Medium freedom (pseudocode or parameterized): Low freedom (specific scripts, exact steps): The test : Ask "if Agent makes a mistake, what's the consequence?" High consequence → Low freedom Low consequence → High freedom D7: Pattern Recognition (10 points) Does the Skill follow an established official pattern? Through analyzing 17 official Skills, we identified 5 main design patterns: Pattern ~Lines Key Characteristics Example When to Use Mindset ~50 Thinking technique, strong NEVER list, high freedom frontend design Creative tasks requiring taste Navigation ~30 Minimal SKILL.md, routes to sub files internal comms Multiple distinct scenarios Philosophy ~150 Two step: Philosophy → Express, emphasizes craft canvas design Art/creation requiring originality Process ~200 Phased workflow, checkpoints, medium freedom mcp builder Complex multi step projects Tool ~300 Decision trees, code examples, low freedom docx, pdf, xlsx Precise operations on specific formats Score Criteria 0 3 No recognizable pattern, chaotic structure 4 6 Partially follows a pattern with significant deviations 7 8 Clear pattern with minor deviations 9 10 Masterful application of appropriate pattern Pattern selection

skill-judge