DAVIS problem analysis including root cause identification, impact assessment, and correlation with other telemetry. Use when querying or investigating detected problems. Trigger: "active problems", "root cause analysis", "problem impact", "affected users", "list problems", "P-12345 details", "recur

Problem Analysis Skill Analyze Dynatrace AI detected problems including root cause identification, impact assessment, and correlation with logs and metrics. Use Cases 1. Active Problem Triage Goal: List and prioritize currently active problems Trigger: "active problems", "what problems are open", "current issues", "availability issues" Done: Prioritized list of active problems with category, user impact, and display IDs 2. Root Cause Investigation Goal: Identify the root cause entity for a specific problem Trigger: "root cause of P 12345", "what caused this problem", "which entity is the root cause" Done: Root cause entity identified with affected entity list and blast radius 3. Problem Trending Goal: Analyze problem patterns over time to identify recurring issues Trigger: "recurring problems", "problem history", "problem trends last 30 days" Done: Trend data showing problem frequency, recurring root causes, and resolution times Overview Dynatrace automatically detects anomalies, performance degradations, and failures across your environment, creating problems that aggregate related alert, warning and info level events and provide root cause and impact insights. What are Problems? Problems are automatically detected, software and infrastructure health and resilience issues that: Automatically correlate related alert, warning, and info level events across services, infrastructure, frontend applications, and user sessions Identify root causes using causal analysis of Smartscape dependencies Assess business impact by tracking affected users and services Reduce alert noise by grouping related symptoms into single problems that share the same root cause and impact Track problem lifecycle from early detection through resolution Event Kinds The event.kind field (stable, permission) identifies the high level event type: event.kind value Description DAVIS EVENT Davis detected infrastructure/application events BIZ EVENT Business events (ingested via API or captured from spans) RUM EVENT Real User Monitoring events AUDIT EVENT Administrative/security audit events event.provider (stable, permission) identifies the event source. Problem Categories Common event.category values: Category Description Example AVAILABILITY Infrastructure or service unavailable Web service returns no data, synthetic test actively fails, database connection lost ERROR Increased error rates beyond baseline API error rate jumped from 0.1% to 15% SLOWDOWN Performance degradation Response time increased from 200ms to 5000ms RESOURCE Resource saturation Container memory at 95%, causing OOM kills CUSTOM Custom anomaly detections Business KPI (orders/minute) dropped below threshold Problem Lifecycle ACTIVE : Currently occurring issues requiring attention CLOSED : Resolved issues used for historical analysis Essential Fields Common Field Name Mistakes ❌ WRONG ✅ CORRECT Description title event.name Problem title/description status event.status Problem lifecycle status severity event.category Problem type/category start event.start Problem start time Correct Status Values Key Fields Reference Standard Query Pattern Always start problem queries with this foundation: Key components: fetch dt.davis.problems The problems data source not(dt.davis.is duplicate) Filter out duplicate detections event.status == "ACTIVE" Show only active problems Time range Always specify a reasonable window Common Query Patterns Active Problems by Category High Impact Active Problems (affecting many users) High Impact Active Problems (affecting many smartscape entities) Specific Problem Details Service Specific Problem History Root Cause Analysis Patterns Basic Root Cause Query Root Cause by Entity Type Identify which entity types most frequently cause problems: Affected entity is an AWS resource Infrastructure Root Cause with Service Impact Problem Blast Radius Calculate entity impact per root cause: Recurring Root Causes Identify entities repeatedly causing problems: Cause Category vs. Root Cause Entity These are different questions — pick the right approach: "What causes problems?" / "most common cause" → Summarize by event.category (SLOWDOWN, ERROR, RESOURCE, AVAILABILITY, CUSTOM). Explain what triggers each category. "Which entity causes problems?" / "root cause entity" → Group by root cause entity name . Lists specific services, hosts, or apps. Cause category breakdown (use when asked about common causes, patterns, or types): Then for each category, explain what triggers it using the Problem Categories table and cite specific entities from the tenant data as examples. Problem Trending and Pattern Analysis Track problem trends over time, identify recurring issues, and analyze resolution performance. Primary Files: references/problem trending.md Timeseries analysis and pattern detection Common Use Cases: Active problems over time with makeTimeseries Problem creation rate by category Recurring problem detection by schedule Resolution time trends and P95 duration analysis Key Techniques: makeTimeseries vs bin() : Choose the right approach for lifecycle spans vs discrete events NULL handling : Use coalesce(event.end, now()) for active problems Peak hours analysis : Identify when problems occur most frequently Impact trending : Track user impact changes over time See references/problem trending.md for complete query patterns and best practices. Cross Domain Problem Queries Problems Associated with Kubernetes Clusters Use affected entity ids or dt.smartscape source.id to find problems related to Kubernetes: Alternative: expand affected entities and filter for K8s entity types: Simple Problem Listing List all problems from the last 24 hours (common request): Response Construction Problem Cause Summaries When summarizing problem causes, categories, or patterns, provide a comprehensive breakdown across all standard categories present in the data: AVAILABILITY, ERROR, SLOWDOWN, RESOURCE, and CUSTOM. For each category found: 1. Category name and count of problems 2. What triggers it — brief explanation (e.g., RESOURCE = CPU/memory/disk threshold exceeded; AVAILABILITY = service or entity became unreachable) 3. Specific examples from the tenant's data (affected entity names, problem IDs) Do not stop after the first two categories — users expect the full picture. Reference the Problem Categories table above for trigger descriptions. Analysis Results When presenting query results: Include entity names (not just IDs) — but choose the efficient method: Few entities (< 5): get entity name calls are fine Many entities: Use query problems tool which returns names directly, or include root cause entity name / entityName() in the DQL query to resolve names inline. Avoid calling get entity name in a loop for 10+ entities — this can exhaust the tool call limit and return no answer at all. Provide actionable recommendations aligned to the identified causes Organize by frequency or impact for easy prioritization Best Practices Essential Rules 1. Always filter duplicates : Use not(dt.davis.is duplicate) to avoid counting the same problem multiple times 2. Use correct status values : "ACTIVE" or "CLOSED" , never "OPEN" 3. Specify time ranges : Always include time bounds to optimize performance 4. Include display id : Essential for problem identification and linking 5. Test incrementally : Add one filter or field at a time when building queries 6. Filter early : Apply not(dt.davis.is duplicate) immediately after fetch Query Development Start simple : Begin with basic filtering, then add complexity Test fields first : Run with limit 1 to verify field names exist Use meaningful time ranges : Too broad wastes resources, too narrow misses data Document problem IDs : Always capture and store display id for reference Root Cause Verification Always filter isNotNull(root cause entity id) when required Cross reference events using dt.davis.event ids Consider time delays: root cause may appear in logs minutes before problem Time Range Guidelines Absolute Timeframes Require Double Quotes When using absolute ISO 8601 timestamps for from and to in DQL queries, always wrap them in double quotes . Unquoted timestamps are a syntax error. Troubleshooting Problem Cause Solution No problems returned Using event.status == "OPEN" Use "ACTIVE" or "CLOSED" — "OPEN" does not exist Duplicate problems in results Missing deduplication filter Add filter not(dt.davis.is duplicate) immediately after fetch Wrong field name ( title , status , severity ) SQL like naming Use event.name , event.status , event.category — see field name table above root cause entity id is null Not all problems have identified root causes Add filter isNotNull(root cause entity id) when querying root causes Query scans too much data / times out Missing time range Always specify from:now() <duration on the fetch command affected entity ids is empty array Problem has no mapped affected entities Check dt.smartscape.service or dt.smartscape source.id as alternatives When to Load References Load [problem trending.md](references/problem trending.md) when: Analyzing problem frequency over time Detecting recurring problems on a schedule Calculating resolution time trends and P95 durations Comparing problem creation rates by category Load [problem correlation.md](references/problem correlation.md) when: Correlating problems with logs or other telemetry Investigating events that preceded a problem Linking problems to deployment or config changes Load [impact analysis.md](references/impact analysis.md) when: Assessing business impact (affected users, services) Calculating blast radius for a root cause entity Prioritizing problems by technical and user impact References [problem trending.md](references/problem trending.md) — Problem trending and timeseries analysis patterns [problem correlation.md](references/problem correlation.md) — Correlating problems with logs and other telemetry [impact analysis.md](references/impact analysis.md) — Business and technical impact assessment [problem merging.md](references/problem merging.md) — When and why DAVIS merges events into problems Related Skills dt dql essentials Core DQL syntax and query structure for problem queries dt obs logs Correlate problems with application and infrastructure logs dt obs tracing Investigate problems through distributed trace analysis

dt-obs-problems