utility-pm-skill-validate
Audits an existing pm-skills skill against structural conventions and quality criteria. Produces a structured validation report with pass/fail checks, severity-graded findings, and actionable recommendations. Use when checking whether a skill meets repo standards before shipping or after making chan
By product-on-purpose · 605 installs
npx skills add product-on-purpose/pm-skills --skill utility-pm-skill-validate
Source repository · Upstream listing
PM Skill Validate
This skill audits an existing pm skills skill against the repo's structural
conventions and quality criteria. It produces a validation report that a
human can scan and that utility pm skill iterate can consume as input.
The validator checks two tiers:
Tier 1 (Structural) deterministic checks that mirror CI: frontmatter,
naming, file presence, description word count.
Tier 2 (Quality) LLM assessed coherence checks: does the output
contract reference the template? Is the example complete? Are checklist
items testable?
When to Use
After creating a skill with utility pm skill builder , before shipping
After manually editing a skill, to confirm it still passes conventions
Before running utility pm skill iterate , to identify what needs improvement
When a convention changes, to audit which skills need updating (batch mode)
When reviewing a contributed skill for quality and completeness
When NOT to Use
To create a new skill from scratch use utility pm skill builder
To fix or improve a skill use utility pm skill iterate (feed it this report)
To run CI checks in a pipeline use scripts/lint skills frontmatter.sh
(this skill is for interactive, deeper than CI validation)
To audit the whole catalog rather than one skill use utility pm skill auditor . This skill goes deep on a single skill; the auditor goes wide across the repo and aggregates cross cutting findings.
Instructions
When asked to validate a skill, follow these steps:
Step 1: Identify the Target
Accept the skill name in any form:
Directory name: deliver prd
Full path: skills/deliver prd/SKILL.md
Slash command: deliver prd
Resolve to the canonical directory path: skills/{name}/ .
If the skill directory does not exist, report immediately:
Batch mode: If the input is all , run Tier 1 structural checks
across all skills and produce a summary table (see Step 5). Do not run
Tier 2 in batch mode.
Step 2: Read Skill Files
Read all files in the skill directory:
File Required Purpose
SKILL.md yes Frontmatter + instructions
references/TEMPLATE.md yes Output template
references/EXAMPLE.md yes Worked example
HISTORY.md no Version history (if present)
evals/trigger fixtures.json no Routing eval fixtures (B 4); present for roster skills
evals/output scenarios/ .md no Output quality scenarios (B 7); present for evaled skills
Also read:
The corresponding command file: commands/{command name}.md
The AGENTS.md entry for this skill
If reading files is not possible (MCP/embedded environment), ask the user
to paste the content of each file before proceeding (see Degraded Mode).
Step 3: Run Tier 1 Structural Checks
Run these deterministic checks. Each produces a PASS or FAIL line.
Check ID What to check Pass condition
frontmatter at byte zero First line of SKILL.md, references/TEMPLATE.md, references/EXAMPLE.md is exactly Line 1 is the opening fence with no preceding HTML comment, BOM, or whitespace. Severity: FAIL. Reference: library/skill output samples/SAMPLE CREATION.md Section 5.
name match Frontmatter name matches directory name Exact string match
description present Frontmatter description exists Non empty value
description length Description word count 20 100 words
version present Frontmatter version exists Non empty, valid SemVer
updated present Frontmatter updated exists Non empty, ISO date
license present Frontmatter license exists Non empty value
phase classification Phase/classification consistency Domain has phase: , foundation/utility has classification: , not both
template exists references/TEMPLATE.md exists File present
template sections TEMPLATE.md has sufficient structure ≥3 level 2 headers
example exists references/EXAMPLE.md exists File present
command exists Command file exists in commands/ File present and references correct skill path
agents entry AGENTS.md has an entry for this skill Entry exists with matching Path:
eval trigger fixtures Routing fixtures present + well formed (B 4) If evals/trigger fixtures.json exists: schema:1 , skill matches dir, =16 queries, =8 per class (trigger / no trigger), =2 near misses. If absent: INFO (not in the trigger eval roster, or fixtures not yet authored). Authoritative gate: scripts/check trigger fixtures.mjs .
eval output scenarios Output scenarios present + well formed (B 7) If evals/output scenarios/ .md exist: each has scenario / skill / family frontmatter, skill matches dir, family maps to an existing docs/internal/eval rubrics/{family}.md , body =100 chars. If absent: INFO (not yet output evaled). Authoritative gate: scripts/check output eval assets.mjs .
reciprocal boundary pointers Declared collision pairs cross point (C 5) If the skill appears in COLLISION PAIRS ( scripts/check trigger fixtures.mjs ): its When NOT to Use names each partner AND each partner's When NOT to Use names it back. FAIL if a declared pair is one directional. If not in any pair: PASS. Authoritative gate: scripts/check reciprocal boundary pointers.mjs .
Step 4: Run Tier 2 Quality Checks
Run these LLM assessed checks. Each produces a PASS , WARN , or INFO
line. Tier 2 findings are capped at WARN unless objectively grounded
(placeholder leakage is the exception it can FAIL ).
Check ID What to assess How to assess Max severity
output contract coverage SKILL.md references the template Check for explicit reference to references/TEMPLATE.md or "use the template" in an Output section. Accept either pattern as valid. WARN only if template is not referenced at all. WARN
checklist verifiability Quality checklist items are testable Read each checklist item. Flag items that are vague ("is good quality") vs. specific ("metrics are measurable"). WARN if ≥2 items are vague. WARN
example completeness EXAMPLE.md fills all template sections Compare headers in TEMPLATE.md against headers in EXAMPLE.md. WARN if EXAMPLE.md is missing sections that appear in the template. Also check for unresolved placeholders. Line count is informational only report it but do not gate on it. WARN
template example alignment EXAMPLE.md follows TEMPLATE.md structure Compare section header ordering. WARN if EXAMPLE.md has sections in a different order or uses different header names than TEMPLATE.md. WARN
description actionability Description tells when to use the skill Check for a trigger phrase like "Use when..." or "Use for..." in the frontmatter description. WARN if the description only says what the skill does without indicating when to use it. WARN
instruction clarity Instructions are numbered and imperative Check for Step headings or a numbered list pattern in the Instructions section. WARN if instructions are prose paragraphs without clear step structure. WARN
placeholder leakage No leftover scaffolding in any shipped file Scan SKILL.md, TEMPLATE.md, and EXAMPLE.md for: [Placeholder] or [Feature Name] patterns, <! ... HTML comments (except the license header), template guidance blockquotes that should have been removed, and authoring notes like "TODO" or "FIXME". FAIL if any are found this is objectively grounded. FAIL
when not to use "When NOT to Use" section present and names neighbors Check for a When NOT to Use section that names the skill's nearest neighbors (when to use them instead). WARN if the section is absent, or present but generic (no neighbor named). This became a convention in the v2.26.0 rewrites and underpins the reciprocity gate; reciprocal boundary pointers (Tier 1) is the deterministic backstop for declared collision pairs. WARN
Quality standard framing: These checks validate against current library
conventions what the shipped library actually does today. Findings graded
WARN or INFO represent the v2.8 quality standard that newer skills (built
with utility pm skill builder ) meet. Older skills may legitimately receive these
findings until iterated through the lifecycle.
Heading and dialect conventions: for the authoritative section skeleton per
dialect (classic, contract shaped, tool family) and the exact canon heading
spellings, see [Skeleton Canon: The Three Sanctioned
Dialects](../../site/src/content/docs/guides/creating pm skills.md skeleton canon the three sanctioned dialects)
in the skill authoring guide. A skill whose headings or section set drift from
its dialect's canon should be flagged in Recommendations even where no Tier 1
check above already covers it; scripts/check heading canon.mjs is the eventual
deterministic backstop, shipping advisory until the full catalog is normalized.
Step 5: Produce the Validation Report
Assemble the report using this exact structure. F 11 ( utility pm skill iterate )
parses this report by section headings and pipe delimited fields.
Report rules:
Result = worst severity found: any FAIL → FAIL , else any WARN → WARN , else PASS .
Structural Checks : one line per Tier 1 check. STATUS is PASS or FAIL .
Quality Checks : one line per Tier 2 check. STATUS is PASS , WARN , or INFO .
Recommendations : only include checks that did NOT pass. Each recommendation
includes the check ID, the target file path, and a specific action.
If all checks pass, the Recommendations section should say: "No issues found."
Omit passing checks from Recommendations only list findings that need action.
Batch mode output (when input is all ):
Run Tier 1 structural checks only across all skills. Produce a summary table:
Degraded Mode
If you cannot read skill files directly (e.g., running via MCP or in an
embedded environment without file system access):
1. Ask the user to provide the content of each required file:
skills/{name}/SKILL.md
skills/{name}/references/TEMPLATE.md
skills/{name}/references/EXAMPLE.md
2. Run all checks against the provided content.
3. Note in the report: "Validated from user provided content (file system not available)."
4. Batch mode is not available in degraded mode single skill only.
Output Contract
The validator MUST produce a validation report following the format in Step 5.
The report:
Uses the exact section headings: Summary , Structural Checks ,
Quality Checks , Recommendations
Uses pipe delimited check lines: STATUS TIER CHECK ID message
Uses pipe delimited recommendations: STATUS CHECK ID Target: path
followed by Action: description on the next line
Includes Report schema: v1 in the header for F 11 compatibility
Includes the Tier 2 caveat line in the Summary section
Quality Checklist
Before delivering the report, verify:
[ ] All Tier 1 structural checks were run (not skipped), including the eval asset checks ( eval trigger fixtures , eval output scenarios , reciprocal boundary pointers ); absent eval assets are INFO, not FAIL, except a one directional declared collision pair which FAILs
[ ] All Tier 2 quality checks were run (not skipped) single skill mode only
[ ] Report follows the exact section and line format from Step 5
[ ] Every non passing check appears in Recommendations with a target file path
[ ] Result field reflects the worst severity found
[ ] Tier 2 findings are capped at WARN (except placeholder leakage which can FAIL)
[ ] No Tier 2 check was marked FAIL unless objectively grounded
Examples
See references/EXAMPLE.md for a completed validation report demonstrating
both Tier 1 and Tier 2 checks against a real shipped skill.