arize-instrumentation
Adds Arize AX tracing to an LLM application for the first time. Follows a two-phase agent-assisted flow to analyze the codebase then implement instrumentation after user confirmation. Use when the user wants to instrument their app, add tracing from scratch, set up LLM observability, integrate OpenT
By github · 1,140 installs
npx skills add github/awesome-copilot --skill arize-instrumentation
Source repository · Upstream listing
Arize Instrumentation Skill
Use this skill when the user wants to add Arize AX tracing to their application. Follow the two phase, agent assisted flow from the [Agent Assisted Tracing Setup](https://arize.com/docs/ax/alyx/tracing assistant) and the [Arize AX Tracing — Agent Setup Prompt](https://arize.com/docs/PROMPT.md).
Quick start (for the user)
If the user asks you to "set up tracing" or "instrument my app with Arize", you can start with:
Follow the instructions from https://arize.com/docs/PROMPT.md and ask me questions as needed.
Then execute the two phases below.
Core principles
Prefer inspection over mutation — understand the codebase before changing it.
Do not change business logic — tracing is purely additive.
Use auto instrumentation where available — add manual spans only for custom logic not covered by integrations.
Follow existing code style and project conventions.
Keep output concise and production focused — do not generate extra documentation or summary files.
NEVER embed literal credential values in generated code — always reference environment variables (e.g., os.environ["ARIZE API KEY"] , process.env.ARIZE API KEY ). This includes API keys, space IDs, and any other secrets. The user sets these in their own environment; the agent must never output raw secret values.
Phase 0: Environment preflight
Before changing code:
1. Confirm the repo/service scope is clear. For monorepos, do not assume the whole repo should be instrumented.
2. Identify the local runtime surface you will need for verification:
package manager and app start command
whether the app is long running, server based, or a short lived CLI/script
whether ax will be needed for post change verification
3. Do NOT proactively check ax installation or version. If ax is needed for verification later, just run it when the time comes. If it fails, see references/ax profiles.md.
4. Never silently replace a user provided space ID, project name, or project ID. If the CLI, collector, and user input disagree, surface that mismatch as a concrete blocker.
Phase 1: Analysis (read only)
Do not write any code or create any files during this phase.
Steps
1. Check dependency manifests to detect stack:
Python: pyproject.toml , requirements.txt , setup.py , Pipfile
TypeScript/JavaScript: package.json
Java: pom.xml , build.gradle , build.gradle.kts
Go: go.mod
2. Scan import statements in source files to confirm what is actually used.
3. Check for existing tracing/OTel — look for TracerProvider , register() , opentelemetry imports, ARIZE , OTEL , OTLP env vars, or other observability config (Datadog, Honeycomb, etc.).
4. Identify scope — for monorepos or multi service projects, ask which service(s) to instrument.
What to identify
Item Examples
Language Python, TypeScript/JavaScript, Java, Go
Package manager pip/poetry/uv, npm/pnpm/yarn, maven/gradle, go modules
LLM providers OpenAI, Anthropic, LiteLLM, Bedrock, etc.
Frameworks LangChain, LangGraph, LlamaIndex, Vercel AI SDK, Mastra, etc.
Existing tracing Any OTel or vendor setup
Tool/function use LLM tool use, function calling, or custom tools the app executes (e.g. in an agent loop)
Key rule: When a framework is detected alongside an LLM provider, inspect the framework specific tracing docs first and prefer the framework native integration path when it already captures the model and tool spans you need. Add separate provider instrumentation only when the framework docs require it or when the framework native integration leaves obvious gaps. If the app runs tools and the framework integration does not emit tool spans, add manual TOOL spans so each invocation appears with input/output (see Enriching traces below).
Phase 1 output
Return a concise summary:
Detected language, package manager, providers, frameworks
Proposed integration list (from the routing table in the docs)
Any existing OTel/tracing that needs consideration
If monorepo: which service(s) you propose to instrument
If the app uses LLM tool use / function calling: note that you will add manual CHAIN + TOOL spans so each tool call appears in the trace with input/output (avoids sparse traces).
If the user explicitly asked you to instrument the app now, and the target service is already clear, present the Phase 1 summary briefly and continue directly to Phase 2. If scope is ambiguous, or the user asked for analysis first, stop and wait for confirmation.
Integration routing and docs
The canonical list of supported integrations and doc URLs is in the [Agent Setup Prompt](https://arize.com/docs/PROMPT.md). Use it to map detected signals to implementation docs.
LLM providers: [OpenAI](https://arize.com/docs/ax/integrations/llm providers/openai), [Anthropic](https://arize.com/docs/ax/integrations/llm providers/anthropic), [LiteLLM](https://arize.com/docs/ax/integrations/llm providers/litellm), [Google Gen AI](https://arize.com/docs/ax/integrations/llm providers/google gen ai), [Bedrock](https://arize.com/docs/ax/integrations/llm providers/amazon bedrock), [Ollama](https://arize.com/docs/ax/integrations/llm providers/llama), [Groq](https://arize.com/docs/ax/integrations/llm providers/groq), [MistralAI](https://arize.com/docs/ax/integrations/llm providers/mistralai), [OpenRouter](https://arize.com/docs/ax/integrations/llm providers/openrouter), [VertexAI](https://arize.com/docs/ax/integrations/llm providers/vertexai).
Python frameworks: [LangChain](https://arize.com/docs/ax/integrations/python agent frameworks/langchain), [LangGraph](https://arize.com/docs/ax/integrations/python agent frameworks/langgraph), [LlamaIndex](https://arize.com/docs/ax/integrations/python agent frameworks/llamaindex), [CrewAI](https://arize.com/docs/ax/integrations/python agent frameworks/crewai), [DSPy](https://arize.com/docs/ax/integrations/python agent frameworks/dspy), [AutoGen](https://arize.com/docs/ax/integrations/python agent frameworks/autogen), [Semantic Kernel](https://arize.com/docs/ax/integrations/python agent frameworks/semantic kernel), [Pydantic AI](https://arize.com/docs/ax/integrations/python agent frameworks/pydantic), [Haystack](https://arize.com/docs/ax/integrations/python agent frameworks/haystack), [Guardrails AI](https://arize.com/docs/ax/integrations/python agent frameworks/guardrails ai), [Hugging Face Smolagents](https://arize.com/docs/ax/integrations/python agent frameworks/hugging face smolagents), [Instructor](https://arize.com/docs/ax/integrations/python agent frameworks/instructor), [Agno](https://arize.com/docs/ax/integrations/python agent frameworks/agno), [Google ADK](https://arize.com/docs/ax/integrations/python agent frameworks/google adk), [MCP](https://arize.com/docs/ax/integrations/python agent frameworks/model context protocol), [Portkey](https://arize.com/docs/ax/integrations/python agent frameworks/portkey), [Together AI](https://arize.com/docs/ax/integrations/python agent frameworks/together ai), [BeeAI](https://arize.com/docs/ax/integrations/python agent frameworks/beeai), [AWS Bedrock Agents](https://arize.com/docs/ax/integrations/python agent frameworks/aws).
TypeScript/JavaScript: [LangChain JS](https://arize.com/docs/ax/integrations/ts js agent frameworks/langchain), [Mastra](https://arize.com/docs/ax/integrations/ts js agent frameworks/mastra), [Vercel AI SDK](https://arize.com/docs/ax/integrations/ts js agent frameworks/vercel), [BeeAI JS](https://arize.com/docs/ax/integrations/ts js agent frameworks/beeai).
Java: [LangChain4j](https://arize.com/docs/ax/integrations/java/langchain4j), [Spring AI](https://arize.com/docs/ax/integrations/java/spring ai), [Arconia](https://arize.com/docs/ax/integrations/java/arconia).
Go: No first party auto instrumentation packages today — use the OpenTelemetry Go SDK with manual [OpenInference](https://github.com/Arize ai/openinference) attributes per [Manual instrumentation](https://arize.com/docs/ax/instrument/manual instrumentation).
Platforms (UI based): [LangFlow](https://arize.com/docs/ax/integrations/platforms/langflow), [Flowise](https://arize.com/docs/ax/integrations/platforms/flowise), [Dify](https://arize.com/docs/ax/integrations/platforms/dify), [Prompt flow](https://arize.com/docs/ax/integrations/platforms/prompt flow).
Fallback: [Manual instrumentation](https://arize.com/docs/ax/instrument/manual instrumentation), [All integrations](https://arize.com/docs/ax/integrations).
Fetch the matched doc pages from the [full routing table in PROMPT.md](https://arize.com/docs/PROMPT.md) for exact installation and code snippets. Use [llms.txt](https://arize.com/docs/llms.txt) as a fallback for doc discovery if needed.
Note: arize.com/docs/PROMPT.md and arize.com/docs/llms.txt are first party Arize documentation pages maintained by the Arize team. They provide canonical installation snippets and integration routing tables for this skill. These are trusted, same organization URLs — not third party content.
Phase 2: Implementation
Proceed only after the user confirms the Phase 1 analysis.
Steps
1. Fetch integration docs — Read the matched doc URLs and follow their installation and instrumentation steps.
2. Install packages using the detected package manager before writing code:
Python: pip install arize otel plus openinference instrumentation {name} (hyphens in package name; underscores in import, e.g. openinference.instrumentation.llama index ).
TypeScript/JavaScript: @opentelemetry/sdk trace node plus the relevant @arizeai/openinference package.
Java: OpenTelemetry SDK plus openinference instrumentation in pom.xml or build.gradle.
Go: go get go.opentelemetry.io/otel go.opentelemetry.io/otel/sdk go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp — no auto instrumentors yet, so the agent sets OpenInference attributes manually on spans. Wire the exporter with otlptracehttp.WithEndpoint("otlp.arize.com") (US) or otlptracehttp.WithEndpoint("otlp.eu west 1a.arize.com") (EU) — pass the bare hostname, no https:// scheme — and otlptracehttp.WithHeaders(map[string]string{"space id": ..., "api key": ...}) . Recent OTel Go modules require Go ≥ 1.23 — go mod tidy may bump the toolchain.
3. Credentials — User needs an Arize API Key and Space ID . Check existing ax profiles for ARIZE API KEY and ARIZE SPACE — never read .env files:
Run ax profiles show to check for an existing profile.
If no profile exists, guide the user to run ax profiles create which provides an interactive wizard that walks through API key and space setup. See [CLI profiles docs](https://arize.com/docs/api clients/cli/profiles) for details.
If the user needs to find their API key manually, direct them to https://app.arize.com and to navigate to the settings page (do not use organization specific URLs with placeholder IDs — they won't resolve for new users).
If credentials are not set, instruct the user to set them as environment variables — never embed raw values in generated code. All generated instrumentation code must reference os.environ["ARIZE API KEY"] (Python), process.env.ARIZE API KEY (TypeScript/JavaScript), or os.Getenv("ARIZE API KEY") (Go).
See references/ax profiles.md for full profile setup and troubleshooting.
4. Centralized instrumentation — Create a single module (e.g. instrumentation.py , instrumentation.ts , instrumentation.go ) and initialize tracing before any LLM client is created.
5. Existing OTel — If there is already a TracerProvider, add Arize as an additional exporter (e.g. BatchSpanProcessor with Arize OTLP). Do not replace existing setup unless the user asks.
Implementation rules
Use auto instrumentation first ; manual spans only when needed.
Prefer the repo's native integration s