Regression testing strategies for AI-assisted development. Sandbox-mode API testing without database dependencies, automated bug-check workflows, and patterns to catch AI blind spots where the same model writes and reviews code. Use when adding regression coverage to AI-assisted code, or when the sa

AI Regression Testing Testing patterns specifically designed for AI assisted development, where the same model writes code and reviews it — creating systematic blind spots that only automated tests can catch. When to Activate AI agent (Claude Code, Cursor, Codex) has modified API routes or backend logic A bug was found and fixed — need to prevent re introduction Project has a sandbox/mock mode that can be leveraged for DB free testing Running /bug check or similar review commands after code changes Multiple code paths exist (sandbox vs production, feature flags, etc.) The Core Problem When an AI writes code and then reviews its own work, it carries the same assumptions into both steps. This creates a predictable failure pattern: Real world example (observed in production): The pattern: sandbox/production path inconsistency is the 1 AI introduced regression. Sandbox Mode API Testing Most projects with AI friendly architecture have a sandbox/mock mode. This is the key to fast, DB free API testing. Setup (Vitest + Next.js App Router) Test Helper for Next.js API Routes Writing Regression Tests The key principle: write tests for bugs that were found, not for code that works . Testing Sandbox/Production Parity The most common AI regression: fixing production path but forgetting sandbox path (or vice versa). Integrating Tests into Bug Check Workflow Custom Command Definition The Workflow Common AI Regression Patterns Pattern 1: Sandbox/Production Path Mismatch Frequency : Most common (observed in 3 out of 4 regressions) Test to catch it : Pattern 2: SELECT Clause Omission Frequency : Common with Supabase/Prisma when adding new columns Pattern 3: Error State Leakage Frequency : Moderate — when adding error handling to existing components Pattern 4: Optimistic Update Without Proper Rollback Strategy: Test Where Bugs Were Found Don't aim for 100% coverage. Instead: Why this works with AI development: 1. AI tends to make the same category of mistake repeatedly 2. Bugs cluster in complex areas (auth, multi path logic, state management) 3. Once tested, that exact regression cannot happen again 4. Test count grows organically with bug fixes — no wasted effort Quick Reference AI Regression Pattern Test Strategy Priority Sandbox/production mismatch Assert same response shape in sandbox mode High SELECT clause omission Assert all required fields in response High Error state leakage Assert state cleanup on error Medium Missing rollback Assert state restored on API failure Medium Type cast masking null Assert field is not undefined Medium DO / DON'T DO: Write tests immediately after finding a bug (before fixing it if possible) Test the API response shape, not the implementation Run tests as the first step of every bug check Keep tests fast (< 1 second total with sandbox mode) Name tests after the bug they prevent (e.g., "BUG R1 regression") DON'T: Write tests for code that has never had a bug Trust AI self review as a substitute for automated tests Skip sandbox path testing because "it's just mock data" Write integration tests when unit tests suffice Aim for coverage percentage — aim for regression prevention

ai-regression-testing