Test AI/LLM features that ship in your product. Covers prompt regression testing, response quality evaluation, tool-call validation, hallucination and RAG grounding checks, nondeterministic-output strategies, red-team/safety scans, eval frameworks, and agent-as-target injection (indirect injection v

<objective AI features fail differently from deterministic software. The same input produces different outputs, correctness is subjective, and failure modes include hallucination, prompt injection, and silent quality decay. A chatbot that confidently cites a fabricated URL passes every toBeDefined() check. This skill covers how to test AI features rigorously despite nondeterminism: versioned prompts, eval suites scored against golden datasets, statistical and property based assertions, tool call validation, grounding checks, and red team safety scans. </objective Quick Route Situation Go to Prompt changed, need to catch quality regressions Prompt Regression Testing → references/prompt regression.md Run the same prompt across providers/models and compare Cross Provider Regression → references/tooling evals.md Score open ended output (relevance/completeness/safety) Response Quality Evaluation → references/eval framework.md Agent calls tools/functions — verify selection and args Tool Call Validation → references/tooling evals.md Output is nondeterministic and exact match keeps flaking Nondeterminism Strategies AI states facts / cites sources / runs over RAG Hallucination & Grounding Pre launch jailbreak, injection, PII, system prompt leak AI Safety Testing → references/tooling evals.md An AGENT (test harness, coding agent) reads tool output / RAG / scan reports / logs Agent as Target Injection → references/injection detector.md Discovery Questions Check .agents/qa project context.md first. If it exists, use it as context and skip questions already answered there. AI features under test: What AI features exist? (Chat, summarization, classification, code gen, recommendations, search) — determinism expectations differ per type. Which provider/model? (Anthropic, OpenAI, Google, open source) — drives the eval harness and red team backend. Are prompts hardcoded, template based, or dynamically constructed? — only versioned prompts are regression testable. Is RAG involved, and what is the knowledge source? — RAG needs grounding tests, not just output checks. Determinism requirements: Which outputs must be deterministic (classification, extraction) vs. creative (chat)? — decides exact match vs. property vs. statistical assertions. What temperature runs in production? — test there, not at 0 "to make tests pass." Can the output be constrained to a JSON schema? — schema constrained extraction is testable with a plain validator, no LLM judge. Quality requirements: How is quality defined? (Accuracy, relevance, completeness, safety, tone) — these become eval metrics with weights and thresholds. Is there a golden dataset of inputs and acceptable outputs? — the anchor for every regression check. Who evaluates quality today? (Humans, metrics, nobody) — if an LLM judges, it must be calibrated against humans. Safety requirements: Does the AI process user generated input? — prompt injection surface. Are there content policy or PII constraints, or a regulated domain? — drives the red team probe set. Core Principles 1. Nondeterminism is inherent, not a bug. LLMs are stochastic; the same prompt yields different outputs across runs. Assert on properties and boundaries, never exact strings. expect(output).toBe("The answer is 42") breaks the next run when the model says "42 is the answer." 2. Test properties, not exact outputs. A good assertion asks: does the response contain the required information, stay in the length range, exclude prohibited content, match the expected format? When you can constrain output to a JSON schema, do that and validate with a plain schema validator — it converts a statistical check into a deterministic one. 3. Evals are the test suite for AI. An eval defines inputs, runs them through the system, and scores outputs against quality criteria. Invest in evals the way you invest in test infrastructure: version them, run them in CI, gate merges on them. 4. Safety testing is non negotiable. AI can produce harmful content, leak the system prompt, echo PII, or be steered by adversarial input. Safety tests are the security tests of AI features — run them on every prompt change, and red team before launch. 5. A judge you have not calibrated proves nothing. LLM as judge scales evaluation, but a judge that disagrees with humans just launders a wrong answer. Measure judge vs human agreement on a labeled held out set and set a minimum bar before trusting it (see Response Quality Evaluation). Tooling Pick the layer that fits the job. Don't reach for hand rolled TS unless none of these match. Tool Best for Notes Promptfoo Prompt regression, A/B + cross provider tests, redteam scans OSS Apache 2.0 CLI; YAML defined suites; MCP target support. Acquired by OpenAI (Mar 2026) but stays open source under its current license + public repo; de facto default for production LLM teams. https://github.com/promptfoo/promptfoo DeepEval Pytest style LLM + agent evals; tool call metrics Current 4.x ships an agent native workflow (run eval → see metric failures inline → patch → retry) that fits Claude Code / Cursor loops. ToolCorrectnessMetric , ArgumentCorrectnessMetric , TaskCompletionMetric cover the tool call patterns this skill teaches. https://github.com/confident ai/deepeval Ragas RAG specific eval (faithfulness, context precision/recall, answer relevance) Use faithfulness for the grounding/hallucination check below. https://github.com/explodinggradients/ragas TruLens Production tracing + RAG triad + non LLM feedback Has deterministic, non LLM feedback functions (e.g. schema/regex checks) that are cheaper than an LLM judge for structured outputs. https://github.com/truera/trulens Inspect AI Government backed agent eval harness; large pre built eval catalog UK AI Security Institute. Date based releases ( release/2025 11 28 ). https://inspect.aisi.org.uk Garak Adversarial prompt scanner / red team probes NVIDIA, Apache 2.0. v0.15.0 current (May 2026): added multi turn GOAT, Agent Breaker (tool aware), system prompt extraction, ModernBERT refusal detector. Probe modules: encoding , dan , promptinject , latentinjection , leakreplay . Run garak list probes for the live set. https://github.com/NVIDIA/garak PyRIT Microsoft AI Red Team's orchestration framework Orchestrated multi turn attacks; complements Garak. https://github.com/Azure/PyRIT Braintrust Commercial evals + prompt playground Hosted, paid; SDK works alongside any of the above. Public benchmarks (HELM, LMSYS Chatbot Arena, Inspect AI's catalog) are for model selection , not app regression — they don't know your domain. Use them when picking a base model; use the tools above for everything after. For runnable entry points — DeepEval tool call validation, the Promptfoo cross provider YAML suite, and the Garak red team command — see references/tooling evals.md . Prompt Regression Testing Version prompts like code Prompts drive your application's behavior; version, review, and test them like code. A versioned prompt object carries its version , template , typed parameters , and a changelog . See references/prompt regression.md for the SUMMARIZE PROMPT object. Baseline response quality Establish a quality baseline per prompt and detect regressions when the prompt, model, or parameters change. Each eval case pairs an input with criteria ( maxLength , mustContain , mustNotContain , sentenceCount , formatCheck ); the test asserts every applicable criterion. See references/prompt regression.md for the baseline eval suite. A/B test prompts When changing a prompt, run both versions against the eval suite over N runs and compare aggregate scores (mean, stddev, min) to pick a winner — or declare a tie when the gap is below threshold. See references/prompt regression.md for the A/B harness. Cross provider regression The same versioned prompt can degrade silently when you switch models or run a fallback provider. Run one prompt across multiple providers in a single Promptfoo suite and assert the same criteria hold on each — this catches a provider that drops a required fact or ignores a length constraint. See references/tooling evals.md for the cross provider YAML (one prompt, providers: [anthropic:..., openai:...] , shared assertions). Response Quality Evaluation Eval framework with weighted metrics For open ended output, score each response on weighted, thresholded metrics and require a passing weighted sum AND every metric over its own floor. Typical wiring: Relevance (weight 0.3, threshold 0.7): LLM as judge rates relevance 0–10, normalized to 0–1. Completeness (weight 0.3, threshold 0.6): compare to a reference answer. Safety (weight 0.4, threshold 1.0): pattern match for prohibited content — must be perfect. A test passes only if every metric clears its threshold and the weighted sum clears the overall bar. See references/eval framework.md for the runnable scorer (per metric scoring functions + weightedSum ). Calibrate the judge before trusting it Any LLM as judge metric needs a calibration gate. On a held out set of cases you have also labeled by hand, score judge vs human agreement (Cohen's kappa, or simple accuracy for binary pass/fail). Set a minimum bar — e.g. kappa ≥ 0.6 — and refuse to ship the judge below it. Re run calibration whenever you change the judge model or the rubric. An uncalibrated judge can rubber stamp wrong answers. See references/eval framework.md for the calibration check. Prefer schema validation over a judge when you can If the output is structured (extraction, classification, function arguments), constrain it to a JSON schema using the provider's structured output mode (Anthropic structured outputs, OpenAI Structured Outputs response format: json schema ) and validate with Zod or Pydantic. This makes extraction near deterministic and lets you drop the statistical assertion entirely — a schema validator is cheaper, faster, and more reliable than an LLM judge for anything with a fixed shape. Golden datasets A golden dataset is a curated set of inputs with known good reference outputs — the most reliable anchor for regression testing. Each case includes: input , reference output , acceptance criteria ( mustContainFacts , mustNotContain , formatRequirements , maxLength ), and metadata ( category , difficulty ). For pulling production prompts into the dataset on a schedule, see observability driven testing . Tool Call Validation When AI systems use tools (function calling, API calls, DB queries), test the selection and invocation logic. DeepEval's metrics are the cleanest entry point — but mind which metric uses a reference: ToolCorrectnessMetric is reference based — it compares tools called to the expected tools you supply. This is where the golden tool list belongs. ArgumentCorrectnessMetric is referenceless and LLM based — it judges whether the arguments make sense given the input; it does NOT consume expected tools . Don't expect it to compare against your reference arguments. TaskCompletionMetric scores whether the agent actually accomplished the task end to end. See references/tooling evals.md for the DeepEval run with the metric wiring annotated. Verify correct tool selection Assert the agent picks the right tool (and arguments) for a query, falls through to search for factual queries, and calls no tools for conversational turns. See references/test patterns.md for the tool selection suite. Argument validation Test that arguments are correctly typed and formatted. A "last week" query should produce valid ISO date strings spanning ~7 days. Also test sanitization: a query containing "; DROP TABLE users; must not reach

ai-system-testing