Use after writing tests to assess coverage quality across structural, mutation, requirements, and API/integration dimensions; organized knowledge for choosing and interpreting coverage analyses.

Test Coverage Analysis A reference manual for choosing, applying, and interpreting test coverage analyses on an existing test suite. This skill is a knowledge reference , not a procedure. It does not tell you when to write tests or which test types to design — that is the job of design testing strategy . It tells you, once tests exist, which mechanical signal best measures what those tests do (and do not) exercise , and how to read that signal honestly. What Coverage Analysis Is Test coverage analysis is the post hoc measurement of how thoroughly a test suite exercises a software artifact along one or more axes . It answers the question "what did my tests actually touch?" — for some specific definition of "touch." The word "coverage" is overloaded. It can mean any of: Structural / code coverage — which lines, statements, branches, conditions, or paths in the source code were executed (measured by instrumentation). Mutation coverage — what proportion of deliberately injected source faults the test suite detects (measured by re running the suite against mutated code). Requirements / feature coverage — which acceptance criteria, user stories, or specification clauses have at least one verifying test (measured by traceability). API / integration coverage — which endpoints, methods, status codes, contract interactions, and schema fields are exercised (measured by request/response inspection). Specification domain coverage — equivalence classes, boundary values, parameter combinations, state transitions, error paths (measured by analyzing test inputs against a model). Category correction: coverage is not a test type Mutation testing, MC/DC, branch coverage, RTM linkage, contract coverage, and schema coverage are measurements about an existing test suite. They are not test types in the way unit, integration, e2e, contract, or smoke tests are. "Should I write a unit test or a mutation test?" is a malformed question. The correct framing is: "I already have unit/integration tests; should I additionally run mutation analysis against them?" Mutation tools generate variants of the source and re execute the existing suite. They produce a score, not new tests. MC/DC and branch coverage are reports computed from instrumented runs of the existing suite. RTM linkage is a property of test metadata (tags, IDs), not a separate execution. If a "test strategy" places mutation testing alongside unit / integration / e2e, that strategy has confused what to test with how to measure the tests . The two questions are orthogonal. The asymmetry principle Low coverage is strong evidence of weak testing. High coverage is weak evidence of strong testing. Coverage is necessary but not sufficient . A test can execute a line without asserting anything meaningful; 100% line coverage is routinely achievable with zero assertions ([thinkinglabs.io](https://thinkinglabs.io/articles/2022/03/19/the fallacy of the 100 code coverage.html), [codeintelligently.com](https://codeintelligently.com/blog/ai generated tests false confidence)). Use coverage as a tripwire , not a trophy . Once a coverage percentage becomes a target, it ceases to be a good metric (Goodhart's law applied to testing; see [Optivem Journal](https://journal.optivem.com/p/code coverage targets recipe for disaster)). What coverage analysis is NOT NOT a measure of test quality. Lines can execute without assertions. NOT a measure of correctness. Coverage proves the test ran, not that it would have failed on a bug. NOT a synonym for "well tested". Mutation testing routinely refutes 100% coverage with no assertions suites. NOT a substitute for risk based test selection per [ISO/IEC/IEEE 29119](https://en.wikipedia.org/wiki/ISO/IEC 29119). NOT a target. Treat as a floor and a trend, never as the goal itself. Per Type Structure Every coverage type in this skill is documented in the same six sub fields, in this order: 1. Definition — what it measures. 2. What it does NOT measure — its limits / blind spots. 3. Typical tools — per ecosystem. 4. When to use vs skip — applicability heuristics. 5. Targets / thresholds & pitfalls — defensible numeric ranges (always with the risk caveat) and common gaming patterns. 6. Cost benefit ROI — order of magnitude cost vs the signal you actually buy. Scan any section by these headings. Structural / Code Coverage Measured by instrumenting the compiled or interpreted program and recording which structural elements (lines, statements, branches, conditions, paths) the test suite executes. Line / Statement Coverage Definition. Percentage of source code lines (or statements) executed at least once. What it does NOT measure. Whether branches were taken in both directions. Whether assertions verified the result. Whether boundary values were tested. Multiple statements on one line distort the metric ([Metridev](https://www.metridev.com/en/metrics/statement vs branch coverage understanding the difference/)). Typical tools. Ecosystem Tool JS/TS [Istanbul / nyc](https://istanbul.js.org/) (built into Jest, Vitest, Karma); coverage flag Python [Coverage.py](https://coverage.readthedocs.io/) + pytest cov ; supports branch mode JVM [JaCoCo](https://www.eclemma.org/jacoco/) — bytecode instrumentation, industry standard C/C++ gcov / lcov / gcovr , [llvm cov](https://llvm.org/docs/CommandGuide/llvm cov.html) Go go test cover , go tool cover ([build cover](https://go.dev/doc/build cover) added integration test mode in Go 1.20) .NET [Coverlet](https://github.com/coverlet coverage/coverlet) (open source default), [JetBrains dotCover](https://www.jetbrains.com/dotcover/), AltCover. OpenCover is in maintenance mode — prefer Coverlet / dotCover / AltCover ([NDepend guide](https://blog.ndepend.com/guide code coverage tools/)) Ruby [SimpleCov](https://github.com/simplecov ruby/simplecov) Swift / Obj C Xcode built in (llvm cov backend) Rust cargo llvm cov , cargo tarpaulin Report formats Cobertura XML, LCOV, Clover; aggregators: Codecov, Coveralls, SonarQube When to use vs skip. Always on; cost is near zero (a CI flag). Never use as a quality goal in itself. Targets / thresholds & pitfalls. 70–85% is typical for general purpose code ([Qt blog](https://www.qt.io/quality assurance/blog/is 70 80 90 or 100 code coverage good enough)). Apply only with the risk caveat (see Risk Based Interpretation below). Tests with no assertions still count lines as covered. Single line if (x) doA(); else doB(); shows 100% statement coverage with only one branch exercised. Snapshot only tests inflate numbers without verifying behavior. Cost benefit ROI. Very high — cost near zero, value is a tripwire on regression in test reach. Branch / Decision Coverage Definition. Percentage of decision branches (true/false outcomes of if , while , for , ?: , switch cases) executed. What it does NOT measure. Compound condition independence ( A && B taken true might never test A=true, B=false ). Order of evaluation. Loop iteration counts. Assertion strength. Typical tools. Same as line coverage; enable with branch (coverage.py), branch mode (Istanbul is branch aware by default), JaCoCo reports branches natively. Strictly stronger than line/statement coverage ([Graph AI](https://www.graphapp.ai/blog/statement coverage vs branch coverage a comprehensive comparison)). When to use vs skip. Default for any non trivial logic. Prefer branch over line as the primary structural metric. Targets / thresholds & pitfalls. 70–80% branch is a "respectable" target for business apps ([Lead With Skills](https://www.leadwithskills.com/blogs/test coverage metrics lines branches conditions paths)) — always paired with the risk caveat. Compound conditions hide gaps: if (A B) achieves 100% branch coverage with only one true evaluating sub condition and one false branch overall. Cost benefit ROI. High — best single structural metric for general purpose code ([LinearB](https://linearb.io/blog/what is branch coverage)). Condition Coverage Definition. Every Boolean sub condition in every decision has taken both true and false at least once. What it does NOT measure. Whether each sub condition independently affects the outcome (that is MC/DC). Does not require all combinations. Typical tools. Same toolchains as branch coverage; many report condition coverage as a separate column. Ecosystem Tool JVM JaCoCo (condition counters in branch reports) C / C++ gcov/gcovr ( branch counts ), Qt Coco .NET Coverlet (condition coverage via Cobertura output) When to use vs skip. Informative for code with compound expressions; rarely useful as a CI gate on its own. Targets / thresholds & pitfalls. Achievable without exercising every combination. if (A && B) hits 100% condition coverage with {A=T,B=F} and {A=F,B=T} — neither makes the decision true . Treat as a diagnostic, not a gate. ROI: Medium — useful diagnostically when investigating why branch coverage looks high but bugs persist; not a gate. MC/DC — Modified Condition/Decision Coverage Definition (per [Wikipedia](https://en.wikipedia.org/wiki/Modified condition/decision coverage)): 1. Every entry/exit point invoked at least once. 2. Every decision has taken every outcome at least once. 3. Every condition in a decision has taken every outcome at least once. 4. Each condition has been shown to independently affect the decision's outcome (holding the other conditions fixed). For n conditions, MC/DC is achievable with n+1 to 2n tests via independence pairs — vastly cheaper than the 2^n of exhaustive multiple condition coverage ([LDRA](https://ldra.com/capabilities/mc dc/)). What it does NOT measure. Loop iteration counts, data values, integration paths, assertion strength. Typical tools. LDRA TBvision, Rapita RapiCover, VectorCAST, Razorcat TESSY, [Qt Coco](https://www.qt.io/quality assurance/coco/feature modified condition decision coverage mcdc), Parasoft C/C++test. Mostly commercial — open source MC/DC is rare. When to use vs skip. When mandated by a standard (DO 178C DAL A, ISO 26262 ASIL D, IEC 62304 Class C high risk modules, EN 50128 SIL 4, IEC 61508 SIL 4). Outside regulated domains, branch coverage + mutation testing covers the same intent at lower cost. Targets / thresholds & pitfalls. 100% by definition in regulated domains. Short circuit evaluation in C like languages can make some independence pairs unreachable; compiler optimizations can collapse conditions, so coverage builds must disable optimization — meaning the coverage build binary is not the release build binary, an acknowledged regulatory risk ([Verifysoft](https://www.verifysoft.com/en ISO 26262 Road Vehicles Functional Safety.html)). Cost benefit ROI. Very high cost (specialist toolchain + labor + documentation overhead); high value only where required by law/standard. Function / Method Coverage Definition. Percentage of declared functions/methods invoked at least once. What it does NOT measure. Anything about the bodies of those functions. Typical tools. Reported by most structural coverage tools as a side column. Ecosystem Tool JVM JaCoCo (method counter) .NET Coverlet (methods column) Python coverage.py ( report m granularity), pytest cov JS/TS Istanbul (functions metric in lcov / json summary) When to use vs skip. As a quick "did I forget a module?" check; never as a primary metric. Targets / thresholds & pitfalls. Often deceptively high — many functions are entered by happy path tests with no error path coverage inside. ROI: Low — informational only; useful as a "module forgotten?" tripw

test-coverage