LangSmith tracing and trace debugging for AgentSeek templates. Covers CLI installation, adding tracing to LangGraph apps, querying traces, and inspecting run IO. Use when debugging agent backends, investigating slow traces, or adding observability to a template.

LangSmith Trace Add tracing to your agent and query traces for debugging. Supports Python and TypeScript. [!IMPORTANT] This skill is tuned for AgentSeek template backends (LangGraph + middleware stacks). For general LangSmith concepts, see the upstream [langsmith skills](https://github.com/langchain ai/langsmith skills) repo. [!CAUTION] Never pass api key as a CLI flag or expose API keys in shell commands / tool calls. The CLI reads LANGSMITH API KEY from the environment automatically. Using api key <value leaks secrets into shell history, process listings, and agent tool call logs. Always rely on the environment variable set in your shell profile or .env file. Installation & Setup 1. Install the CLI The binary installs to ~/.local/bin/langsmith . If langsmith is not found after install, add to your shell profile: 2. Set your API key (environment variable only) Add to ~/.zshrc (or the template's .env file) — the user must fill in their own key: Key format must start with lsv2 pt . Get yours at https://smith.langchain.com/settings. The CLI reads this env var automatically — never use api key flags. Do not echo, print, or pass the key value in any shell command or tool call argument. 3. Verify If you see a JSON array of projects, you're set. Common failures: command not found — ~/.local/bin not in PATH (see step 1) 401 Unauthorized — key is wrong or expired; regenerate at LangSmith settings Empty array [] — valid auth but no projects yet; create one in the UI or run a traced app Adding Tracing LangGraph / LangChain apps (automatic) Just set environment variables — no code changes needed: For serverless (Python): also set LANGCHAIN CALLBACKS BACKGROUND=false to ensure traces flush before function exit. Non LangChain apps Use the @traceable decorator (Python) or traceable() wrapper (TypeScript) and wrap your LLM client: Debugging a Trace (Step by Step) This is the recommended workflow for investigating agent behavior: Step 1: Find the right project Look at last run start time to find which project has recent activity. Don't assume — LangGraph apps default to the "default" project, not a named one. Step 2: List recent traces Step 3: Get the trace hierarchy This returns the full run tree — use it to understand the agent's execution flow. Step 4: Get IO for all runs in a trace This gives you inputs and outputs for every run (LLM calls, tool calls, middleware). Step 5: Drill into a specific run AgentSeek Trace Structure Our templates produce traces with this typical hierarchy: Key points: The actual LLM call is always the innermost ChatOpenAI run Tool results are in the <tool name run (e.g., generate cover , execute ) Middleware wrappers are transparent — they add latency but the IO you care about is at the leaf nodes Gotchas full vs include io on individual runs Problem: langsmith run get <id full can return null for inputs/outputs, even though the data exists. Despite full being documented as equivalent to include metadata include io include feedback , the underlying API behavior differs for individual run fetches in some CLI versions. Solution: Always use include io explicitly when inspecting specific runs: full works reliably on trace export and run list , but has inconsistent behavior on individual run get calls. If this is fixed in a future CLI version, include io still works correctly — it's always safe. Null IO even with include io If inputs/outputs come back null even with include io , the project has IO logging disabled. Check for these environment variables in the template's .env : Remove or set to false to enable IO capture for debugging. Project confusion LangGraph apps trace to the "default" project unless LANGSMITH PROJECT is explicitly set. Always run langsmith project list first and check last run start time to find where your traces actually landed. Tips Add format pretty for human readable output during interactive debugging Use LANGSMITH ENDPOINT env var if connecting to a self hosted LangSmith instance The middleware ordering in the trace tree is configurable — your template may differ slightly from the diagram above CLI Reference Full command reference: [reference/cli commands.md](reference/cli commands.md)

langsmith-trace