Adds Arize AX tracing to an LLM application for the first time. Detects the stack, routes to the single matching integration doc, wires auto-instrumentation after user confirmation, and verifies traces land. Use when the user wants to instrument their app, add tracing from scratch, set up LLM observ

Arize Instrumentation Skill Add Arize AX tracing to an app for the first time: detect the stack → fetch the one matching integration doc → wire auto instrumentation → verify a trace lands. Route locally. Map the detected stack to a single doc page via [references/integration routing.md](references/integration routing.md) (exhaustive for tracing integrations) and fetch only that page . When the app uses an agent framework, route on the framework , not the provider SDK it wraps — a bare openai / anthropic import inside a framework app is not the integration target; route to a provider page only when the app calls the provider SDK directly with no framework in play. If the stack isn't listed there, it has no dedicated integration — use [manual instrumentation](https://arize.com/docs/ax/instrument/manual instrumentation). Never bulk fetch the [PROMPT.md](https://arize.com/docs/PROMPT.md)/[llms.txt](https://arize.com/docs/llms.txt) aggregates. Rules: inspect before mutating; tracing is additive, never change business logic; follow existing style; no secrets in code and never ask the user to paste secrets (API keys, tokens) into the chat reference ARIZE API KEY / ARIZE SPACE ID env vars only, set by the user in their own .env /shell; preserve or ask for the app's Arize region/export endpoint instead of assuming US see [references/regions and endpoints.md](references/regions and endpoints.md); ask before persistent local state ( ax profiles, .zshrc , env vars) see [references/ax profiles.md](references/ax profiles.md). Phase 1: Analysis (read only — no code/files) Detect from manifests + imports: language, package manager, LLM providers, frameworks, existing tracing ( TracerProvider , register() , ARIZE / OTEL , Datadog/Honeycomb), existing Arize endpoint/region config ( ARIZE COLLECTOR ENDPOINT , an in code Arize endpoint option, or an OTEL EXPORTER OTLP ENDPOINT confirmed to target Arize), and whether the app runs tools / an agent loop (manual CHAIN/TOOL spans only if the matched framework instrumentor doesn't already cover them — decided in Phase 2). Confirm scope first — a monorepo, multiple services, or multiple frameworks needs a "which one?" question before touching anything; don't pick for the user. Output a short summary (stack, proposed integration, existing tracing, scope). If the target is clear and the user asked to instrument now, continue; if ambiguous or analysis only was requested, stop and confirm. Phase 2: Implementation (after the target is confirmed) 1. Fetch the matched integration doc and follow its install + wiring verbatim. 2. Install with the detected package manager, before writing code — exact packages come from the matched doc. Python base: arize otel (latest 0.13.0 , verify on PyPI) + openinference instrumentation {name} (package hyphens, import underscores). TS/JS: @opentelemetry/sdk trace node + the matched @arizeai/openinference (or a first party exporter, e.g. Mastra's @mastra/arize ). Go has no integration doc — see [references/go.md](references/go.md) for install, wiring, flush, and manual spans. 3. Credentials and region — app needs an API key + Space + exporter endpoint/region. Inspect only the target app's own config; never scan sibling repos/shell files, surface secrets, or accept a pasted key. See [references/credentials and config.md](references/credentials and config.md), [references/regions and endpoints.md](references/regions and endpoints.md), and [references/ax profiles.md](references/ax profiles.md). 4. Centralize init in one module, before any LLM client is created. Existing OTel → add Arize as an additional exporter; don't replace it. Auto vs manual: prefer the auto instrumentor — do not hand roll spans it already covers (duplicates spans, drifts from semconv). Add manual spans only for logic no instrumentor sees, or when the stack has no instrumentor at all. When the app calls the provider SDK directly, the OpenAI and Anthropic SDK instrumentors capture the LLM call — including the model's tool call request (name + args) — but NOT the tool's execution, its return value, or the agent/chain boundary. A raw SDK app with its own tool calling loop MUST add a TOOL span per execution (to capture the result) and a CHAIN (or AGENT) span to group the turn , or those never appear — do this with @tracer.tool / @tracer.chain decorators wherever the functions exist (they set kind, metadata, and status automatically); hand roll spans only where no decoratable function does. Some agent frameworks instead drive the model through their own client layer and emit their own OpenTelemetry spans; for those, a provider instrumentor captures nothing , so instrument via the framework's integration page rather than the provider SDK. Framework instrumentors (LangChain/LangGraph/OpenAI Agents SDK) typically cover tools and chains — verify in the matched doc before skipping manual spans. Keep register() / arize otel go for setup; see [references/manual spans.md](references/manual spans.md) and [manual instrumentation](https://arize.com/docs/ax/instrument/manual instrumentation). Cross cutting (every stack): Project name is required — missing it → HTTP 500 ( service.name alone fails). Set as a resource attribute: Python register(project name=…) ; TS SEMRESATTRS PROJECT NAME / model id ; Go Options{ProjectName} or openinference.project.name . Don't hand roll a TracerProvider /exporter — use register() / arize otel go ; raw OTel only when integrating an existing provider. Order: register tracer → instrumentors → clients. Region: do not assume US. Preserve ARIZE COLLECTOR ENDPOINT or an endpoint already confirmed to target Arize. A generic OTEL EXPORTER OTLP ENDPOINT may belong to an existing non Arize exporter; preserve that exporter separately and ask for the Arize SaaS region instead of reusing it blindly. See [references/regions and endpoints.md](references/regions and endpoints.md). Prefer @tracer. decorators over hand rolled spans — they set kind, input.value / output.value , full TOOL metadata, and terminal status automatically, so they can't emit the UNSET /incomplete spans hand rolled ones do. The decorator is on the function definition , so dynamic dispatch is no reason to hand roll. See [references/manual spans.md](references/manual spans.md). A hand rolled span must set, before exit: openinference.span.kind , input.value / output.value , and for a TOOL span all of tool.name / tool.description / tool.parameters . Most missed line: start as current span records a raised exception as ERROR but never sets OK — so add span.set status(Status(StatusCode.OK)) on the success path or the span exports UNSET and fails scoring. See [references/manual spans.md](references/manual spans.md). Flush before exit (CLI/scripts/notebooks) or async exports drop: Python force flush() + shutdown() , TS shutdown() , Go defer tp.Shutdown(ctx) (never log.Fatalf / os.Exit mid span). See [references/session tracking.md](references/session tracking.md). Sessions: for obvious multi turn interactions (e.g. a multi turn chatbot) or when the user asks, add session.id so turns group into one conversation — see [references/session tracking.md](references/session tracking.md). Never silently override the app's project/space/IDs/endpoint — surface mismatches. Verification Done only when: app builds/typechecks, starts with tracing, emits ≥1 real request, and you confirm the trace in Arize or give a precise app vs Arize blocker. Trigger an LLM call, then use the arize trace skill to confirm spans (kind, input.value / output.value , parent child; CHAIN+TOOL if tools run). No traces → check ARIZE SPACE ID / ARIZE API KEY , init order, the configured collector endpoint/region, exporter logs ( GRPC VERBOSITY=debug ); common causes: wrong region endpoint, missing project name (500), unflushed short lived process, or export/verify credential context mismatch (report it, don't rewrite config — [references/credentials and config.md](references/credentials and config.md)). For the deterministic trace lookup sequence, blocker classification, and the post arrival smoke check, follow [references/verification.md](references/verification.md). After a confirmed trace Emit milestones (install → wiring → run → export → verify); mark recovered errors resolved; end separating done from blockers. Then briefly offer the next step: arize trace (inspect/debug), arize dataset (curate), arize evaluator (evals), arize experiment (compare), arize prompt optimization (improve prompt). Quality issues → arize trace first. References [integration routing](references/integration routing.md) (the router) · [credentials and config](references/credentials and config.md) · [regions and endpoints](references/regions and endpoints.md) · [ax profiles](references/ax profiles.md) · [manual spans](references/manual spans.md) · [go](references/go.md) (Go — no doc page exists) · [session tracking](references/session tracking.md) · [verification](references/verification.md) · [tracing assistant mcp](references/tracing assistant mcp.md).

arize-instrumentation