Produce a multi-quarter QA strategy document. Covers scope, risk-based prioritization, test levels (unit/integration/E2E), pyramid analysis, entry/exit criteria, quality KPIs, tool selection rationale, CI scaling levers, and timeline planning. Output is an actionable strategy document, not a shelf d

<objective Generate an actionable QA strategy tailored to the product, team, and risk profile — a document that drives daily testing decisions, not a compliance artifact that collects dust. A team with 150 E2E tests and a 52 minute pipeline thinks it has good coverage; this skill diagnoses the inverted pyramid, prescribes the rebalance, and ties every element to a measurable KPI. </objective Discovery Questions Before writing a single line of strategy, gather context. Check .agents/qa project context.md first — if it exists, use it as the foundation and skip questions already answered there. Product & Business Context What is the product? (SaaS, e commerce, API platform, mobile app, content site) Who are the users? (consumers, enterprise, internal, developers) What are the business critical flows? (signup, checkout, payment, data export) What is the release cadence? (continuous, weekly, bi weekly, quarterly) What compliance requirements exist? (SOC2, HIPAA, PCI DSS, GDPR, EU AI Act) Current Testing State What test levels exist today, and the current count at each level? What frameworks and tools are in use? Current code coverage, and the target if any? How long does the CI pipeline take end to end? What is the current flakiness rate? Pain Points & Goals Biggest quality pain points? (regressions, slow feedback, flaky tests, gaps) What broke in the last 3 releases? What escaped to production? What does "good enough quality" look like for this team? Appetite for investment in test infrastructure? Team & Constraints Team size and composition (devs, QA, SDET, manual testers) Skill levels with automation tools Budget constraints for tooling Timeline pressure — is there a deadline driving this strategy? Calibrate to team maturity (set team maturity in .agents/qa project context.md ): startup — Minimal pyramid: unit tests + a handful of critical E2E paths. Skip contract testing and formal metrics until CI runs reliably. Phase 1 under 4 weeks. growing — Full pyramid with defined coverage targets, flakiness thresholds, and CI quality gates. Add risk based prioritization. established — SLA backed quality gates, multi environment coverage, advanced tooling (contract testing, chaos, observability), and formal review cadence. Core Principles 1. Risk based prioritization over exhaustive coverage. Not all code is equal — a payment bug costs 1000x more than a tooltip typo. Allocate testing effort proportional to business risk, not code volume. The risk matrix drives where to invest; run risk based testing first if no matrix exists yet. 2. Test pyramid health is the leading indicator. A healthy suite is many fast unit tests, fewer integration, fewest E2E. When the shape inverts (ice cream cone) feedback is slow, maintenance is high, and confidence is paradoxically low. Diagnose the current shape before prescribing anything. 3. Shift left: catch defects earlier. Every defect found later costs exponentially more. Push validation earlier — static analysis before tests, unit before integration, contract before E2E. Design reviews catch architecture bugs no test can find. 4. Every strategy element has a KPI. If you cannot measure it, you cannot improve it. Coverage targets, flakiness thresholds, escape rate goals, MTTR limits — each section names a number and a tracking cadence. 5. Living document, not a shelf document. Reviewed quarterly at minimum. It carries a revision history, a named owner per section, and explicit re evaluation triggers (new product area, team change, major incident, defect escape). Strategy Document Template Walk through each section to produce the final document. Tailor depth to complexity — a 5 person startup needs 5 pages, not 50. The final document follows a 13 section structure (Executive Summary through Revision History); see references/diagrams and worksheets.md for the copy paste markdown skeleton, and references/strategy templates.md for four fully worked examples (SaaS, e commerce, API first, media). 1. Scope & Objectives Define boundaries clearly. Ambiguity here causes gaps and wasted effort downstream. In scope: every product area, service, and integration this strategy covers; functional and non functional types; platforms and browsers/devices. Out of scope: state what is NOT covered and why — third party services tested only at the contract level, legacy systems slated for deprecation. Objectives: 3 5 measurable objectives with timelines, e.g. "Reduce defect escape rate from 12% to under 5% within two quarters," "Achieve 80% unit coverage on all services launched after Q1 2026." 2. Test Levels & Types Define each level, what it covers, who owns it, and expected volume. Level What It Validates Owner Framework Target Count Run Frequency Unit Functions, business logic, edge cases Developers Vitest/Jest/pytest 70 80% of all tests Every commit Integration Service interactions, DB queries, API contracts Developers + QA Supertest/pytest + Testcontainers 15 20% of all tests Every PR E2E Critical user journeys through the full stack QA/SDET Playwright/Cypress 5 10% of all tests Pre deploy + nightly API Contract compliance, schemas, error handling Developers Playwright APIRequestContext/Schemathesis Per endpoint Every PR Visual UI regression, layout shifts, responsive QA Playwright/Argos/Chromatic Key pages Nightly Performance Response times, throughput, resource usage DevOps/QA k6/Lighthouse Critical paths Weekly + pre release Security OWASP Top 10, dep vulns, auth flows Security/DevOps OWASP ZAP/Snyk Per release Pre release + scheduled Accessibility WCAG 2.2 AA, screen reader compat QA/Frontend axe core Key flows Every PR Adjust to what the product actually needs. Not every product needs visual regression. Every product needs unit and integration tests. 3. Test Pyramid Analysis Diagnose the current shape, then define the target. Shapes. The suite takes one of four shapes — healthy pyramid (many unit, few E2E), ice cream cone (inverted, E2E heavy), diamond (integration heavy), or hourglass (unit heavy and E2E sparse with a missing integration middle). Each signals a different feedback/maintenance trade off. See references/diagrams and worksheets.md for the side by side ASCII diagram. Current state. Count tests at each level, compute the percentage split, identify the shape, then capture CI duration, flaky rate, and pass rate. See the Current State Assessment Worksheet in the reference file. Target state. Define target ratios (70 80% unit, 15 20% integration, 5 10% E2E) with concrete counts, plus target CI duration and flaky rate. See the Target State Worksheet. Action plan — if ice cream cone or diamond: 1. Freeze E2E growth — no new E2E tests unless covering a net new critical path. 2. Decompose existing E2E — find E2E tests validating logic testable at unit level (a checkout test asserting tax math becomes a unit test on the tax function), rewrite them down a level. 3. Add unit requirements to the PR checklist — every PR touching business logic ships unit tests. 4. Set CI gates — fail PRs where the unit:E2E ratio drops below threshold. Before rebalancing, separate genuinely flaky E2E tests from ones exposing real bugs — quarantining a flaky test that hides a race condition is how the regression escapes. For flake root cause triage and quarantine mechanics, see test reliability . Action plan — if hourglass: 1. Invest in integration infrastructure — DB fixtures, service stubs, contract tests. 2. Identify service boundaries — each boundary needs integration tests for happy path + error cases. 3. Use contract testing (Pact) for inter service communication. 4. Risk Assessment Matrix Map features to risk levels — this directly determines testing depth. Score each feature as Impact (1 Negligible → 5 Catastrophic) × Likelihood (1 Rare → 5 Almost Certain); the product (1 25) maps to LOW/MED/HIGH/CRIT bands. See references/diagrams and worksheets.md for the full 5x5 matrix with every cell labeled. Risk Level Testing Action Automation Monitoring CRITICAL (15 25) Full automation + manual exploratory + load test Mandatory, every commit Real time alerts, synthetic monitoring HIGH (10 14) Full automation + periodic manual review Mandatory, every PR Dashboard + daily checks MEDIUM (5 9) Automation for happy path + key error cases Recommended Weekly review LOW (1 4) Manual testing or skip Optional None required Example mapping: Feature Area Impact Likelihood Score Testing Approach Payment processing 5 Catastrophic 3 Possible 15 CRIT Automated E2E + unit + contract + monitoring User authentication 5 Catastrophic 2 Unlikely 10 HIGH Automated E2E + security scan + unit Product search 3 Moderate 3 Possible 9 MED Unit + integration + happy path E2E Dashboard rendering 2 Minor 3 Possible 6 MED Unit + visual regression Email preferences 1 Negligible 2 Unlikely 2 LOW Manual verification 5. Environment Strategy Environment Purpose Test Types Data Deploy Trigger Local Developer feedback Unit, integration Mocked/seeded On save CI Automated validation Unit, integration, lint, SAST Ephemeral On push/PR Staging Pre production validation E2E, visual, performance, security Production like (anonymized) On merge to main Production Monitoring & smoke Smoke tests, synthetic monitoring Live On deploy Document: how test data is managed per environment, whether environments are ephemeral (preview deployments) or long lived, who has access, and how environment specific config is managed. 6. Tool Selection Rationale Do not pick tools first. Understand needs, then select tools that fit. Score each candidate against weighted criteria. Criteria (weight) Tool A Tool B Tool C Fits tech stack (25%) Team familiarity (20%) Community & docs (15%) CI integration (15%) Maintenance cost (10%) Speed of execution (10%) License cost (5%) Weighted total Score each 1 5, multiply by weight, sum for the weighted total. Beyond license fees, account for total cost of ownership : setup time (configure CI, write first tests, train team), writing time (time 5 real tests to measure), maintenance time (how often tests break on framework updates), debug time (good error messages cut this), and infrastructure cost (browser farms, parallel runners). Common stack starting points — document why you chose or deviated: Product Type Unit Integration E2E API Visual React SaaS Vitest Testing Library + MSW Playwright Supertest Playwright screenshots Next.js Vitest Testing Library + MSW Playwright Supertest Playwright screenshots Python API pytest pytest + Testcontainers pytest + requests Schemathesis N/A Mobile (RN) Jest Testing Library + MSW Detox / Maestro / Appium 3.x Supertest Appium screenshots Vue SaaS Vitest Testing Library + MSW Playwright Supertest Playwright screenshots AI/LLM features Vitest DeepEval Playwright + Promptfoo evals Promptfoo / Ragas N/A For AI/LLM features, add explicit risk testing for hallucinations, bias, prompt injection, and privacy — see ai system testing and compliance testing (EU AI Act). Reference frameworks: CTAL AT v2.0 (ISTQB, Advanced Agile Tester, 2026) — a new Advanced level certification requiring CTFL v4.0, not an update of an Advanced predecessor; it supersedes the retired Foundation level CTFL AT Agile e

test-strategy