Hybrid fingerprint + LLM pipeline for bug classification, deduplication, and ticket generation. Normalizes CI logs, creates stable fingerprints, clusters near-duplicates, then uses LLM for severity classification and ticket writing. Includes bug reporting templates and severity/priority matrix. Use

<objective A hybrid pipeline for bug classification, deduplication, and ticket generation. Deterministic fingerprinting handles deduplication (what LLMs are bad at); LLM handles explanation, severity assessment, and ticket writing (what LLMs are good at). Key reframe: The LLM is best at explaining and routing, not deduplication. Teach agents to DESIGN the pipeline, not BE the pipeline. </objective Discovery Questions Check .agents/qa project context.md first — it carries tech stack, component mapping, and known flaky areas that improve classification accuracy. Use it and skip anything already answered there. Then clarify: 1. What is the failure source? CI pipeline logs (GitHub Actions, GitLab CI, Jenkins, CircleCI) Test framework output (Playwright, Jest, pytest, Vitest) Production error monitoring (Sentry, Datadog, Bugsnag) Manual bug reports from QA or users 2. What is the ticket destination? Jira, Linear, GitHub Issues, Azure DevOps, Shortcut What fields are required? (component, severity, priority, labels) What workflows exist? (triage board, auto assignment rules) 3. What is the deduplication scope? Same test run? Same sprint? Same release? All time? Do you already have fingerprinting? What is the current duplicate rate? 4. What approval workflow is needed? Auto create tickets with human review? Suggest tickets for human approval before creation? Auto close duplicates? (dangerous require approval) 5. What historical data exists? Past bug reports with resolution data? Flaky test history? Known environment issues? Component ownership mapping? Core Principles 1. Deterministic first, LLM second. Use stable, reproducible fingerprinting for deduplication and clustering. Use LLM only for tasks requiring understanding: severity classification, root cause hypothesis, and human readable ticket writing. 2. Normalize before comparing. Raw CI logs are full of timestamps, port numbers, process IDs, and random suffixes that make identical failures look different. Strip all noise before fingerprinting. 3. Fingerprints are anchored to stable elements. Exception type, top stack frames, test name, error message template, and URL pattern are stable. Timestamps, request IDs, and ephemeral ports are not. 4. Human approval before destructive actions. Auto closing a ticket as duplicate or auto merging reports requires human confirmation. False deduplication wastes more time than manual triage. 5. Classification drives routing. The value of triage is not the label itself but the routing decision it enables: which team, what priority, what SLA. 6. Track triage accuracy. Measure how often auto classification matches human judgment. Below 85% accuracy, the pipeline needs tuning. The Pipeline Step 1: Normalize Strip noise that makes identical failures look different. Normalization rules (apply in order): Example: Rule 5 strips the port but not the literal loopback IP — 127.0.0.1 stays in the fingerprint. That's fine for same host failures, but two runners that bind different hosts (e.g. 127.0.0.1 vs 0.0.0.0 ) will split into separate fingerprints. If you run heterogeneous hosts, add a rule to normalize bind addresses too. Step 2: Extract Stable Anchors From the normalized log, extract elements that identify the failure regardless of environment or timing. Anchor types (in priority order): Anchor Example Stability Exception type TypeError , AssertionError , HTTP 500 Very high Error message template Cannot read property 'X' of undefined High Top 3 stack frames at processOrder (order.ts:142) High Test name checkout.spec.ts completes payment Very high URL pattern POST /api/orders High HTTP status code 500 , 429 , 503 Very high Exit code exit code 1 , SIGKILL High Assertion diff Expected: 200, Received: 500 Medium Extraction rules: Keep function names but strip line numbers (they change with edits) Keep URL paths but strip query parameters and IDs in paths ( /api/orders/<ID ) Keep error message structure but replace dynamic values with placeholders Keep test file and test name exactly as is Step 3: Hash Canonical Form Create a deterministic fingerprint from the extracted anchors. Algorithm: Fingerprint properties: Same failure always produces same fingerprint (deterministic) Different failures produce different fingerprints (collision resistant) Minor log format changes do not change fingerprint (stable) Fingerprint is short enough for Jira labels and GitHub tags Example: Step 4: Cluster Near Duplicates Exact fingerprint matching catches identical failures. Similarity scoring catches related failures that differ slightly (same root cause, different manifestation). Similarity dimensions: Dimension Weight Match Criteria Exception type 0.30 Exact match Error message 0.25 Levenshtein distance < 20% of message length Stack frames 0.25 Jaccard similarity of top 5 frames 0.6 Component/file 0.10 Same directory or module Test name 0.10 Same describe block or test file Clustering threshold: similarity score 0.75 = likely duplicate, suggest merge. Human review required for: Scores between 0.60 and 0.75 (ambiguous) First occurrence of a new fingerprint (no history to compare) Failures in components with known intermittent issues Step 5: LLM Classify After deterministic fingerprinting and clustering, use the LLM to classify the failure. The prompt feeds in exception, message, top 5 stack frames, test name, and CI context, and asks for five fields: 1. Failure category — test bug application bug environment issue flaky test build failure 2. Severity — critical major minor trivial (see the severity matrix below) 3. Component — inferred from stack trace and file paths 4. Suspected root cause — 1 2 sentence hypothesis 5. Confidence — high medium low ; when low, the LLM states what extra information would resolve it Route low confidence classifications to human review rather than auto acting. See references/pipeline prompts and integration.md for the full prompt text and references/classification taxonomy.md for the bug category, severity, and component mapping definitions the prompt should reference. Failure categories (see references/ci failure analysis.md for detail): Category Description Typical Action Application bug The app is broken File bug ticket, assign to owning team Test bug The test is wrong Fix the test, no app change needed Environment issue CI infra / network / service down Retry, notify infra team Flaky test Intermittent, non deterministic Quarantine, investigate root cause Build failure Compilation, dependency, config Fix build, usually blocking Step 6: LLM Generate Ticket Once classified, use the LLM to generate a human quality bug ticket. The prompt takes the classification plus the normalized error, a log excerpt, and related cluster fingerprints, and produces: Title — concise, searchable, includes component name (under 80 chars) Description — what happened, in plain language (never raw logs) Steps to reproduce — derived from the test name and log context Evidence — relevant log lines, assertion diffs, screenshots if available Suggested labels — [component, severity, failure category, fingerprint] Suggested assignee — based on component ownership, if known The fingerprint belongs on the ticket (label and Fingerprint field) so future dedup can match. See references/pipeline prompts and integration.md for the full prompt and the bug report template. Step 7: Human Approval No automated action without review. The pipeline suggests; humans decide. Approval decisions: Create ticket — New failure, clear root cause, assign to team Merge into existing — Duplicate of known issue, add evidence to existing ticket Quarantine test — Flaky test, not an app bug, quarantine and schedule investigation Retry and monitor — Environment issue, retry CI, alert if persists Dismiss — Known issue already fixed in pending deploy, or test bug with obvious fix Severity/Priority Matrix Severity measures impact. Priority measures urgency. They are independent dimensions. Severity Definitions Severity Definition Examples Critical System unusable, data loss, security breach, no workaround Payment processing fails, user data exposed, app crashes on launch Major Core feature broken, degraded experience, workaround exists Search returns wrong results, checkout requires page reload, form data lost on back button Minor Non core feature affected, cosmetic with functional impact Sorting does not persist, tooltip clipped on mobile, secondary action fails Trivial Cosmetic only, no functional impact Typo in label, 1px alignment, inconsistent capitalization Priority Definitions Priority Definition SLA (example) P0 Fix immediately, blocks release or production Same day P1 Fix this sprint, significant user impact This sprint P2 Fix next sprint, moderate impact Next sprint P3 Fix when convenient, low impact Backlog Severity x Priority Decision Guide Critical Major Minor Trivial Affects all users P0 P0 P1 P2 Affects segment ( 10%) P0 P1 P2 P3 Affects few users (<10%) P1 P1 P2 P3 Edge case only P1 P2 P3 P3 Bug Report Template Use the same template for any bug report, whether auto generated or human written. It carries the defect heading, severity/priority/component/environment/fingerprint/reporter metadata, then Description, Steps to Reproduce, Expected/Actual Behavior, Evidence, Frequency, Suggested Root Cause, and Related Issues. See references/pipeline prompts and integration.md for the full copy paste Markdown template. Deduplication Patterns Pattern Detection Action Exact duplicate Same fingerprint Merge into existing ticket, add evidence Near duplicate Same cluster (similarity 0.75) Link tickets, suggest merge for human review Same root cause, different symptom Same exception type + overlapping frames in different tests Create parent ticket linking symptom tickets Regression of fixed bug Fingerprint matches closed ticket Reopen ticket, flag as regression, increase priority Flaky recurrence Same fingerprint intermittently across CI runs Tag as flaky, quarantine if rate 10% CI Failure Analysis See references/ci failure analysis.md for comprehensive patterns. Key decision: consistent failure = test bug or app bug; intermittent failure = flaky test or environment; multiple failures at once = environment or shared component; build failure = code or dependency issue. Integration Patterns The pipeline output is tracker agnostic: Step 6 produces title, description, labels, severity, and component that map to any tracker's fields. See references/pipeline prompts and integration.md for the gh issue create / fingerprint dedup commands, the GitHub Actions "triage on failure" workflow, and notes on Jira/Linear/Azure DevOps REST/GraphQL integration. Buy vs Build Before implementing the full pipeline, check whether a hosted platform already covers the work you'd be doing. Several tools now ship AI driven test triage that overlaps directly with Steps 4 6. Platform Covers Notes Trunk Flaky Tests Fingerprinting, clustering, severity routing, native PR comments + webhooks Dedicated Agents feature for triage; documented Quarantining workflow — the closest off the shelf analog to this skill's pipeline CloudBees Smart

ai-bug-triage