Analyze production Agentforce agent behavior using session traces and Data Cloud, and manage Agent Health Monitoring (AHM) alerts. TRIGGER when: user queries STDM session data or Data Cloud trace records; investigates production agent failures, regressions, or performance issues; asks about session

Agentforce Observability Improve Agentforce agents using session trace data and live preview testing. Three phase workflow: Observe Query STDM sessions from Data Cloud (if available), OR run test suites + preview with local traces as fallback Reproduce Use sf agent preview to simulate problematic conversations live Improve Edit the .agent file directly, validate, publish, verify Platform Notes Shell examples below use bash syntax. On Windows, use PowerShell equivalents or Git Bash. Replace python3 with python on Windows. Replace /tmp/ with $env:TEMP\ (PowerShell) or %TEMP%\ (cmd). Replace jq with python c "import json,sys; ..." if jq is not installed. Routing Gather these inputs before starting: Org alias (required) must be authenticated (else sf org login web ) Agent API name (required for preview and deploy; ask if not provided) Agent file path (optional) path to the .agent file, typically force app/main/default/aiAuthoringBundles/<AgentName /<AgentName .agent . Auto detect if not provided. Session IDs (optional) analyze specific sessions; if absent, query last 7 days Days to look back (optional, default 7) Alert owner user (optional, alerts only) user whose alerts to list/manage; defaults to the current user Determine intent from user input: No specific action run all three analysis phases: Observe surface issues ask if user wants to Reproduce and/or Improve "analyze" / "sessions" / "what's wrong" Phase 1 only, then suggest next steps "reproduce" / "test" / "preview" Phase 2 (run Phase 1 first if no issues in hand) "fix" / "improve" / "update" Phase 3 (run Phase 1 first if no issues in hand) "create alert" / "set up monitoring" / "alert me when" Phase 4 (create) "list alerts" / "show my alerts" / "delete alert" Phase 4 (list / delete) "why isn't my alert firing" / "have my alerts fired" / "alert notifications" Phase 4 (notification count signal + Incidents view + metric verify) Resolve agent name Before any STDM query, resolve the user provided agent name against the org to get the exact MasterLabel and DeveloperName : MasterLabel = display name used by STDM findSessions and Agent Builder UI (e.g. "Order Service") DeveloperName = API name with version suffix used in metadata (e.g. "OrderService v9") The api name flag for sf agent preview/activate/publish uses DeveloperName without the vN suffix (e.g. "OrderService") Store these values: AGENT MASTER LABEL for findSessions() agent filter AGENT API NAME DeveloperName without vN suffix, for sf agent CLI commands PLANNER ID the Salesforce record ID for this agent Locate the .agent file Step 1 Search locally: If the user provided an agent file path, use that directly. Otherwise, search for files matching AGENT API NAME . Step 2 If not found locally, retrieve from the org: Known bug: sf project retrieve start creates a double nested path: force app/main/default/main/default/aiAuthoringBundles/... . Fix it immediately after retrieve: Step 3 Validate the retrieved file: Read the .agent file and verify it has proper Agent Script structure: system: block with instructions: config: block with developer name: start agent or subagent blocks with reasoning: instructions: Each subagent should have distinct instructions: content (not identical across subagents) Store the resolved path as AGENT FILE for Phase 3. Phase 0: Discover Data Space Before running any STDM query, determine the correct Data Cloud Data Space API name. Note: sf api request rest is a beta command do not add json (that flag is unsupported and causes an error). The response shape is: The name field is the API name to pass to AgentforceOptimizeService . Decision logic: If the command fails (e.g. 404 or permission error), fall back to 'default' and note it as an assumption. Filter to only status: "Active" entries. If exactly one active Data Space exists, use it automatically and confirm to the user: "Using Data Space: <name ". If multiple active Data Spaces exist, show the list (label + name) and ask the user which to use. Store the selected name value as DATA SPACE for all subsequent steps. Prerequisite check: STDM DMOs After deploying the helper class (step 1.0), run a quick probe to verify the STDM Data Model Objects exist in Data Cloud: If STDM CHECK:FAIL : STDM is not activated. Inform the user and switch to Phase 1 ALT : STDM (Session Trace Data Model) is not available in this org. To enable: Setup Data Cloud Data Streams and verify "Agentforce Activity" is active. Proceeding with fallback: test suites + local traces. If STDM CHECK:OK , proceed to Phase 1 (STDM path). Phase 1 ALT: Observe Without STDM (Fallback Path) When STDM is not available, use test suites and sf agent preview authoring bundle with local trace analysis. Data source When to use Pros Cons STDM (Phase 1) Historical production analysis Real user data, volume Requires Data Cloud, 15 min lag Test suites + local traces (Phase 1 ALT) Dev iteration, orgs without STDM Instant, full LLM prompt, variable state Preview only, no real user data 1 ALT.1 Run existing test suite (if available) 1 ALT.2 Derive test utterances from .agent file (if no test suite) If no test suite exists, derive utterances: one per non entry subagent (from description: keywords), one per key action, one guardrail test, one multi turn test. 1 ALT.3 Preview with authoring bundle (local traces) Run each test utterance through preview to generate local trace files: Trace file location: .sfdx/agents/{BundleName}/sessions/{sessionId}/traces/{planId}.json 1 ALT.4 Local trace diagnosis Issue type Trace command Subagent misroute jq r '.plan[] \ select(.type=="NodeEntryStateStep") \ .data.agent name' "$TRACE" Action not called jq r '.plan[] \ select(.type=="EnabledToolsStep") \ .data.enabled tools[]' "$TRACE" LOW adherence jq r '.plan[] \ select(.type=="ReasoningStep") \ {category, reason}' "$TRACE" Variable capture fail jq r '.plan[] \ select(.type=="VariableUpdateStep") \ .data.variable updates[]' "$TRACE" Vague instructions jq r '.plan[] \ select(.type=="LLMStep") \ .data.messages sent[0].content' "$TRACE" DefaultTopic trace quirk: With authoring bundle , the root .topic field often shows "DefaultTopic" even when routing works. Always use NodeEntryStateStep.data.agent name for the real subagent chain. Entry answering directly (SMALL TALK pattern): If start agent trace shows SMALL TALK grounding and transition tools visible but none invoked, add "You are a router only. Do NOT answer questions directly." to start agent instructions. 1 ALT.5 Classify and present Classify issues using the categories in references/issue classification.md . After presenting findings, automatically proceed to agent config evidence analysis. Phase 1: Observe Query STDM Full STDM query details, Apex service deployment, and response parsing: see references/stdm queries.md 1.0 Deploy helper class (once per org) Deploy AgentforceOptimizeService Apex class to the org. Check if already deployed first: If not deployed, copy from skill directory and deploy. See references/stdm queries.md for full steps. 1.1 Find sessions Query recent sessions using findSessions() . Parse DEBUG STDM RESULT: from the Apex debug log. If findSessions returns empty, switch to Phase 1 ALT. 1.2 Get conversation details Use getMultipleConversationDetails() for up to 5 sessions (most recent first). Returns turn by turn data with messages, steps, topics, and action results. 1.2b Get LLM prompt/response (optional) When LOW adherence detected, use getLlmStepDetails() to get the actual LLM prompt and response. 1.2c Get aggregated metrics (recommended first step) Use getAggregatedMetrics() for high level health dashboard: session rates, top intents, quality distribution, RAG averages. 1.2d Get moment insights (per session detail) Use getMomentInsights() for intent summaries, quality scores (1 5), and retriever metrics per session. 1.2e Run observability queries (RAG deep dive) Use runObservabilityQuery() for targeted RAG analysis: KnowledgeGap, Hallucination, RetrievalQuality, AnswerRelevancy, Leaderboard. 1.3 Reconstruct conversations Render turn by turn timeline from ConversationData JSON for each session. 1.4 Identify issues Full issue pattern table and classification categories: see references/issue classification.md Check each session for: action errors, subagent misroutes, missing actions, wrong inputs, variable capture failures, no transitions, slow actions, LOW adherence, abandoned sessions, dead subagents, publish drift, dead hub anti pattern, entry answering directly, and safety issues. Voice agents (has modality voice: block): Also check for: Response verbosity — flag any agent response over 3 sentences (voice UX anti pattern; also a silence/nudge timer trigger) Visual formatting in responses — lists, links, markdown that don't render in speech Missing confirmation patterns — actions modifying data without repeating back key details Missing voice wiring — voice agent lacks a VoiceCallId linked variable ( @VoiceCall.Id ) or the connection customer web client: block, or someone added a non existent connection voice: block Latency anti patterns — cross reference trace step durations against the field verified patterns in /agentforce generate [ references/voice latency heuristics.md ](../agentforce generate/references/voice latency heuristics.md): synchronous writes on the live call path, bulky retrieval returned raw to the reasoning LLM, chained external callouts, over decomposed subagent routing, and slow actions with no ack phrase. Latency fixes are flag only unless purely instructional (ack phrase, turn length, spoken form rule). TTS garble / missing spoken form rule — action outputs or responses that surface prices, phone numbers, or IDs without a spoken form instruction rule. Priority: P1 = action errors, misroutes, LOW adherence; P2 = missing actions, variable bugs, knowledge gaps; P3 = performance, abandoned sessions, voice UX issues, voice latency anti patterns. 1.5 Present findings and agent config evidence Present sessions analyzed, issues grouped by root cause category, and uplift estimate. Then automatically proceed to analyze the .agent file to confirm root causes. Full structural analysis checks, cross reference procedures, and publish drift detection: see references/issue classification.md Retrieve the .agent file from the org, run automated checks (subagent count vs action blocks, dead hub detection, orphan actions, cross subagent variable dependencies), and cross reference STDM symptoms against the file structure. Phase 2: Reproduce Live Preview Full preview procedures, trace diagnosis commands, and classification criteria: see references/reproduce reference.md Build one test scenario per confirmed issue from Phase 1. Run each through sf agent preview with authoring bundle (generates local traces). Run each scenario 3 times and classify: Verdict Criteria [CONFIRMED] Same failure in 3/3 runs [INTERMITTENT] Failure in 1 2 of 3 runs [NOT REPRODUCED] Passes in 3/3 runs Only [CONFIRMED] and [INTERMITTENT] issues proceed to Phase 3. Key commands: Run these from the Salesforce project directory. start requires an action mode with authoring bundle ( simulate actions or use live actions ); that flag is rejected by send and end . Trace location: .sfdx/agents/{Name}/sessions/{sessionId}/tr

agentforce-observe