release-readiness
Validate release readiness with evidence-based go/no-go decisions. Covers go/no-go checklists, smoke test suite design, staged rollout validation, rollback criteria and procedures, and post-deployment verification. Ensures release confidence comes from data, not feelings. Use when: "release ready,"
By petrkindlmann · 678 installs
npx skills add petrkindlmann/qa-skills --skill release-readiness
Source repository · Upstream listing
<objective
"I think it's fine" ships the bug that the rollback you never practiced can't undo at 6 PM on a Friday. This skill turns "ready to ship" into something measurable: a go/no go checklist with evidence for each item, a sub 5 minute smoke suite, a staged rollout with metric gated promotion, rollback thresholds defined before deploy, and post deployment verification. Every section gives concrete criteria, not aspirations.
</objective
Discovery Questions
Check .agents/qa project context.md first — if it exists, use it and skip anything answered there. Then ask only what's missing.
Release cadence and process:
How often do you release, and who makes the go/no go call? (Continuous/daily/weekly vs engineering lead/QA lead/release manager — sets how heavyweight the checklist should be.)
Is there a release train schedule or is it ad hoc?
How many environments exist between dev and production? (staging, pre prod, canary)
Current state:
What does the current go/no go process look like, and is it documented?
Has a release ever been rolled back? How long did it take? (Reveals whether rollback is real or aspirational.)
What was the last release incident and its root cause? Any release blocking bugs right now?
Infrastructure and capabilities:
Do you have rollback capability, and how long does a rollback take?
Can you do staged/canary deployments?
Do you have feature flags, and how are they managed? (Decides flag based vs infra level rollout.)
What monitoring and alerting is in place? Are database migrations reversible?
Team and communication:
Who is on call during and after releases?
How are stakeholders notified, and is there a release communication channel?
How are release notes generated?
Core Principles
1. Release confidence comes from evidence, not feelings
"I think it's fine" is not a go/no go criterion. Evidence means: all CI pipelines green, smoke tests pass on staging, performance budgets met, no open P0/P1 bugs. If you can't point to data, you're not ready.
Bind each result to the candidate commit, deployed build, environment, configuration, and
run time. Preserve first attempt failures and retry outcomes. List passed, failed, skipped,
blocked, and unrun checks separately; a green exit can still conceal excluded tests.
Evidence from a prior commit does not establish readiness after a change. An unavailable
payment sandbox means the payment check is blocked, not passed. Record the named approver's
actual decision separately from the QA recommendation; do not infer sign off from optimism.
2. Smoke tests are the last safety net, not the only safety net
Smoke tests catch catastrophic failures. They are not a substitute for thorough testing throughout the development cycle. If your smoke test suite is the only thing between you and production, you have a process problem upstream.
3. Staged rollouts reduce blast radius
Deploying to 100% of users simultaneously means 100% of users are affected by any bug. Staged rollouts (canary, percentage based, ring based) let you catch issues when they affect 1% of users instead of all of them.
4. Rollback criteria must be defined BEFORE release
If you wait until something is on fire to decide whether to roll back, you'll waste critical minutes debating. Define the criteria in advance, relative to baseline: "If error rate exceeds 2x baseline within 15 minutes, we roll back. No discussion needed." Tie the trigger to your DORA targets — a release whose error rate would push you past your change failure rate target, or whose recovery would blow your MTTR target, is one the rollback rule exists to stop.
5. Every release is a learning opportunity
Post deployment verification isn't just about catching bugs. Track what went well, what was slow, what was stressful. Improve the process continuously.
Collect Current Local Evidence
When the full QA Skills checkout is present, run
node <qa skills /scripts/qa.mjs run project <project goal "Check release readiness"
against the candidate's configured checks. Read both report.html and result.json .
Carry every failed, blocked, incomplete, skipped, and unrun area into the release
checklist. The report is evidence for its named checks, not an automatic release approval.
Use risk based testing to identify critical flows absent from the current suite, and
execute those domain workflows before making a go/no go recommendation.
Go/No Go Checklist
Use this as a template. Adapt it to your context. Every item should be verifiable with evidence, not just "I checked." Store the completed checklist as a versioned artifact (e.g. RELEASE <version .md or a tracked issue) so sign off is auditable.
Include an evidence table with check, candidate/build, environment, command or report link,
execution time, result, and outstanding gap. A readiness review can legitimately finish with
a no go recommendation and missing evidence. It does not require deploying or performing
post deployment actions to make the checklist look complete.
Automated Checks (Must Pass)
[ ] All CI pipelines green — Unit tests, integration tests, E2E tests, type checking, linting
[ ] Smoke test suite passes on staging — Critical user journeys verified in the staging environment
[ ] No open P0/P1 bugs for this release — Check issue tracker, filter by milestone/label
[ ] Performance budgets met — API response times and bundle size within thresholds; Lighthouse CI for frontend releases (skip Lighthouse for API/backend only releases — it measures page load, not service health)
[ ] Security scan clean — No high/critical vulnerabilities in npm audit / Snyk / Dependabot
[ ] API contract tests pass — No breaking changes to public APIs
[ ] Visual regression tests pass — No unintended visual changes
[ ] Accessibility checks pass — axe core scan shows no new violations
Manual Checks (Verify Before Go)
[ ] Feature flags reviewed — Document which flags are enabled/disabled in this release; confirm flag states for production
[ ] Monitoring and alerts configured — New features have corresponding alerts (error rate, latency, business metrics)
[ ] Rollback plan documented and tested — Written procedure exists; rollback has been practiced on staging
[ ] Database migrations tested — Tested forward migration; backward migration verified if schema change is reversible
[ ] Third party dependency changes reviewed — New or upgraded external dependencies checked for breaking changes
[ ] Release notes prepared — Changelog updated, stakeholder facing summary written
[ ] On call engineer identified — Named person is available and has context on the release contents
[ ] Communication plan ready — Stakeholders know the release is happening; support team briefed on changes
[ ] No conflicting releases — Other teams aren't deploying simultaneously
[ ] Deploy window confirmed — Not deploying during peak traffic or before a weekend (unless continuous deployment)
Risk Assessment
[ ] Change scope categorized — Small (config change, copy update), Medium (new feature, refactor), Large (architecture change, migration)
[ ] Blast radius estimated — What percentage of users could be affected if something goes wrong?
[ ] Revert complexity assessed — Can this be reverted in <5 minutes? Does reverting require a data migration?
Smoke Test Suite Design
What to Include
Smoke tests cover critical user journeys only . If these fail, the application is fundamentally broken.
Typical smoke test suite (5 8 tests):
1. Application health — Homepage loads, returns 200, no JavaScript errors in console
2. Authentication — User can log in with valid credentials, session is established
3. Core workflow — The primary value delivering action works (e.g., create a document, submit a form, add to cart or complete a purchase flow)
4. Data retrieval — Key data loads correctly (dashboard populates, search returns results, product page loads)
5. Payment/transaction (if applicable) — Payment flow completes with test credentials
6. API health — Primary API endpoints return valid responses with correct schemas
7. Navigation — Critical navigation paths work (deep links, redirects, menu items)
8. Error handling — Application shows a user friendly error page for invalid routes (404)
What NOT to Include
Edge cases (those belong in regression tests)
Visual perfection (that belongs in visual regression tests)
Performance benchmarks (that belongs in performance tests)
Exhaustive form validation (that belongs in unit/integration tests)
Keeping It Fast
Target: under 5 minutes for the entire smoke suite.
Run tests in parallel where possible
Use API calls instead of UI interactions for setup (create test user via API, not through registration form)
Skip non critical assertions (don't check exact copy text, check that elements exist)
Use a dedicated test account with pre created data (don't create data from scratch each run)
Avoid unnecessary waits — use smart waiting (wait for element, not sleep(3000) / waitForTimeout )
Environment Specific Smoke Tests
Staging smoke tests:
Full smoke suite (all 5 8 tests)
Can use test payment providers
Can test with feature flags in upcoming release configuration
Can test database migrations
Production smoke tests:
Subset of staging smoke tests (3 5 tests)
Use synthetic test accounts (clearly labeled, won't affect analytics)
Never test with real payment transactions (use sandbox mode or skip)
Focus on: app loads, auth works, core read operations work, API responds
Post deployment smoke tests:
Run immediately after deploy completes (within 60 seconds)
Same as production smoke tests
If any fail, trigger alert and begin rollback evaluation
Staging is not production: it has different data volumes, traffic patterns, third party configurations, and infrastructure scale. That gap is exactly why production and post deployment smoke tests exist on top of staging verification.
Staged Rollout Validation
Rollout Stages
A typical staged rollout. The same ladder expressed for flag based rollouts adds a 25% step (see below):
Stage Traffic % Duration Purpose
Canary 1% 15 30 min Catch crashes, exceptions, obvious failures
Early adopters 10% 1 2 hours Validate error rates, latency, business metrics
Partial rollout 25 50% 2 4 hours Confirm stability at scale
Full rollout 100% — Monitor for 24 hours post deployment
What to Monitor Between Stages
Before promoting to the next stage, verify all of these:
Error metrics:
Error rate (HTTP 5xx) is not higher than baseline
Exception count is not higher than baseline
No new error types appearing in logs
Performance metrics:
P50 and P95 latency are within acceptable range (relative to baseline, not an absolute ceiling)
No increase in timeout errors
Database query times are stable
Business metrics:
Conversion rate is not dropping
User engagement (page views, actions) is stable
Revenue/transaction volume is normal (if applicable)
Infrastructure metrics:
CPU and memory usage are normal
No increase in queue depth or message backlog
No disk space issues from new logging
Automated Promotion Criteria
Define rules for automatic promotion between stages. Each gate combines an error rate ceiling, a latency ceiling expressed relative to baseline , a stability window, and (at higher stages) business metric guardrails. See references/rollout automation.md for the full canary→10%→50%→100% promotion ruleset.
Feature Flag Gradual Rollout
An alternative to infrastructure level canary deploys:
1. Deploy new code to 100% with the feature flag OFF
2. Enable the flag for internal users first (dogfooding)
3. Enable for 1% of users (canary equivalent)
4. Gradually increase: 10%, 25%, 50%, 100%
5. Rem