Build and visualize QA dashboards and reports with Allure Report, Grafana, and ReportPortal. Covers test execution visualization, stakeholder-facing quality reports, trend/flakiness panels, release-readiness gates, alerting, and CI integration for automated report generation. Use when: "test dashboa

<objective Build dashboards that drive decisions, not dashboards that display data. The failure mode this prevents: a wall of panels showing "5 failures" with no link to the failures, no target, and no trend — a notification dressed up as a tool that everyone stops opening by week three. This skill delivers audience specific QA reporting — rich HTML reports (Allure), real time trend panels with regression alerts (Grafana/InfluxDB), self hosted AI assisted aggregation (ReportPortal), and automated stakeholder summaries — each panel mapped to a question someone actually asks and every red indicator drilling down to the failing test. </objective Quick Route Situation Go to Already on Grafana for infra metrics Grafana Dashboards — add test metrics alongside prod Test runner has a hosted dashboard (Cypress/Playwright) SaaS Native Dashboards — least plumbing Need rich HTML report, no infra to run Allure Report — generate in CI, publish artifact Need self hosted aggregation across frameworks + AI triage ReportPortal Need a single view across multiple runners Grafana or Allure (cross runner aggregation) Need a weekly summary / release verdict for stakeholders Stakeholder Reports Discovery Questions Check .agents/qa project context.md first — if it exists, use it and skip anything answered there. Then: 1. What tool do you use for test reporting today? Console output, JUnit XML, HTML reports, or a dedicated platform? Identifies the starting point and how much plumbing is left. 2. Who will look at the dashboard? Developers need failure details and traces; QA leads need trends and flakiness; leadership needs release confidence and defect rates. Each audience needs a different view. 3. What decisions should the dashboard drive? "Is this build safe to release?" "Which tests need fixing?" "Is quality improving sprint over sprint?" Dashboards without a decision context become shelfware. 4. Where do test results live? GitHub Actions artifacts, S3, a database? The storage location determines which dashboard tool is practical. 5. What CI platform? GitHub Actions, GitLab CI, Jenkins? Each has different artifact and reporting integrations. Core Principles 1. Dashboards answer questions, they do not display numbers. Every panel must map to a question someone actually asks. "What is the flakiness rate?" is a question. "Total test count" is trivia. 2. Different audiences need different views. A developer debugging a CI failure needs stack traces, screenshots, and traces. A VP needs a single number: "Are we ready to release?" Do not force both through the same dashboard. 3. Real time for CI, trends for leadership. CI dashboards update on every pipeline run. Leadership dashboards aggregate weekly or per sprint. Mixing cadences confuses both audiences. 4. Drill down to action. Every red indicator must link to the specific failing test, the specific flaky test, or the specific coverage gap. A dashboard that shows "5 failures" but does not link to the failures is a notification, not a tool. 5. Automate report generation. Reports that require manual effort (running scripts, copying data, formatting slides) will not survive the first busy sprint. Generate reports from CI pipelines automatically. Allure Report Allure generates rich HTML reports from test results with history, categories, and retries built in. It works with Playwright, Jest, Vitest, pytest, and most frameworks. Allure 2 vs Allure 3. Two paths, and they are easy to mix up because the framework adapters ( allure playwright , allure vitest , allure jest ) emit the same Allure 2 result files; only the reader differs: Allure 2 path — allure commandline (2.42.1, Jun 2026). brew install allure installs this , not v3. Commands: allure generate / allure open / allure serve ; categories via a categories.json dropped into allure results/ . Stable, frozen, dependency bump only now. Allure 3 path — allure npm package + allurerc.mjs (3.9.0, May 2026). TypeScript rewrite: plugins, single file config, real time allure watch , project wide quality gates, multi environment reports, and Allure Service for server side history. Commands: npx allure run / npx allure generate / npx allure watch ; categories move into allurerc.mjs (the dropped in categories.json is a v2 concept). Choose Allure 3 for new projects. If you follow only the brew install allure / allure generate commands you are on the v2 path — that is fine and fully supported; just know which one you are running. Minimal Playwright reporter wiring — register the adapter as reporter: ["allure playwright", { outputFolder, environmentInfo }] so every run drops results into allure results/ with the environment captured: Add per test metadata ( allure.severity / feature / story / tag ) to drive grouping, define failure categories to split product bugs from infra breakage, and preserve history/ across CI runs so trends exist at all. Without history an Allure report is a single snapshot — no trends, no intermittent failure detection. See references/allure.md for the full Playwright/Vitest configs, the v2 categories.json , the v3 allurerc.mjs + allure run runnable path, and the GitHub Actions history preservation steps. Grafana Dashboards Grafana gives real time dashboards with alerting. Best when the team already runs Grafana for infrastructure and wants test metrics next to production metrics. Data pipeline: a post test CI step parses results (JUnit XML, coverage JSON, timing) and pushes points to a time series DB (InfluxDB or a Prometheus pushgateway); Grafana queries those. The push script writes two measurements — test execution (one point per test, tagged by suite / test name / status / branch / run id ) and test run summary (one point per run with pass rate , total , failed , avg duration ms ). Tag every point with branch and run id so panels can filter to main and link back to a specific run. The script runs under if: always() , so wrap the write loop in try/flush/close — a throw mid loop otherwise loses every buffered point. See references/grafana.md for the full push test metrics.ts (with the flush guard) and the GitHub Actions step. Recommended panels Panel Question Query shape Pass Rate Trend (time series) Is quality improving? SELECT mean("pass rate") FROM "test run summary" WHERE "branch"='main' GROUP BY time(1d) , thresholds at 95% (yellow) / 99% (green) Release Readiness (stat) Is main ready to release? SELECT last("pass rate") FROM "test run summary" WHERE "branch"='main' , red <95 / yellow 95–99 / green ≥99 — pair with a coverage stat ≥80 Flakiness Top 10 (table) Which tests waste the most time? SELECT "test name", count("retries") AS retry count FROM "test execution" WHERE "retries" 0 AND time now() 14d GROUP BY "test name" ORDER BY retry count DESC LIMIT 10 CI Duration Trend Is the pipeline getting slower? avg duration ms over time with a target line at 600s Full queries plus Coverage Trend and Duration Distribution panels are in references/grafana.md . Alerting Provision alert rules as YAML under provisioning/alerting/ (Grafana 11+). The one the Done When requires: main pass rate drops more than 2 percentage points in a single day. Build it from two queries (mean pass rate over the last 1d vs the day before) feeding a math expression $yesterday $today 2 , then route via a Slack contact point (incoming webhook URL). The full provisioned rule + contact point is in references/grafana.md . Also alert on: pass rate below 95% (10m window), CI duration above 15 min, coverage drop 2% in a week. Dashboards are for investigation; alerts are for detection — a dashboard no one opens catches nothing. ReportPortal Self hosted test reporting platform with ML powered failure analysis, cross framework aggregation, and real time dashboards. As of 26.0.3 ReportPortal also ingests agentic test results (launches carry an AGENTIC vs AUTOMATION execution type badge) — relevant if part of your suite runs through Claude Code or another agent. Playwright integration Install with npm i D @reportportal/agent js playwright . Feature What it does Auto analysis ML failure classification: product bug, test bug, system issue, or to investigate Defect type mapping Custom defect categories with sub types for your project Flaky test detection Tests that flip pass/fail across launches Merge launches Combine sharded CI runs into one unified view Quality gates Pass/fail criteria per launch (max failures, min pass rate) Comparison Side by side of two launches to spot regressions Quality gates are queryable after a run ( GET /api/v1/$PROJECT/launch/$LAUNCH ID/quality gate ) — use the status as a CI gate and fail the pipeline if it is not PASSED . Allure TestOps (managed alternative) If self hosting feels heavy, Allure TestOps (26.2.x line, 2026) is the SaaS path: Allure 3 quality gates, named environments, global attachments, and Allure 3 style flaky detection (flags a test once it shows ≥3 status transitions across its last 10 runs). Its MCP server is in public beta (26.1.1) , letting AI agents query launches and quality gates directly — relevant when your QA workflow runs through Claude Code / Cursor. SaaS Native Test Dashboards If your test runner has a first class hosted dashboard, prefer it over Allure/Grafana for that runner's native data — less plumbing, more retention, built in PR comments. Cross pollinate with Allure/Grafana only for cross runner aggregation. Platform Test runner Native data + PR comments Cypress Cloud Cypress Test replay, parallelization, flake detection; AI add on (Auto Heal, Bug Triage) Currents.dev Cypress, Playwright OSS friendly Cypress Cloud alternative; lower price point Playwright HTML + reporter=blob Playwright Free, self hosted; combine shards with merge reports Datadog Test Optimization Any (CI side) Flaky Test Management (now with Bits AI auto fix), TIA, native APM Allure TestOps Any Allure 3 quality gates, named environments, MCP server beta Combining sharded Playwright runs (free, native). Have each shard emit a blob report, then merge into one HTML report — the no cost answer to "combine sharded CI runs": Use Allure or Grafana when you need one dashboard across multiple runners, or when a SaaS option's pricing/data residency does not fit. Otherwise the SaaS native dashboard is usually the cheapest path to PR level signal. Stakeholder Reports Weekly QA Summary — automate via a scheduled CI job. Include: pass rate + trend, new vs fixed failures, top 5 flaky tests, coverage delta, avg CI duration. Classify health: STABLE ( = 98%), NEEDS ATTENTION ( = 95%), CRITICAL (< 95%). Post to Slack automatically. Release Quality Report — generate before each release. Gate on: E2E pass rate = 99%, unit pass rate 100%, branch coverage = 80%, zero critical bugs, major bugs <= 2, and the Core Web Vitals budget. Output a READY / NOT READY verdict with a per gate pass/fail breakdown. Core Web Vitals "good" thresholds for the perf gate (current as of 2026): LCP < 2500ms, INP < 200ms, CLS < 0.1. Gate on INP, not FID — INP replaced FID as a Core Web Vital on 2024 03 12 and FID was fully retired on 2024 09 09. Recommended Dashboard Panels A practical set covering the most common questions teams ask. Panel Question It Answers Data Source Audience Pass/Fail Trend Is quality improving or degrading? CI test results over time Everyone Flakiness Top 10 Which tests waste the most time? Tests with retries in last 14 days Developers, QA Coverage Heatmap Where are we blind? Coverage

qa-dashboard