Audit experiment integrity before claiming results. Uses cross-model review (external reviewer backend) to check for fake ground truth, score normalization fraud, phantom results, and insufficient scope. Use when user says "审计实验", "check experiment integrity", "audit results", "实验诚实度", or after expe

Experiment Audit: Cross Model Integrity Verification 🔒 Do not wrap this skill in /loop , /schedule , or CronCreate . It is verdict bearing — it judges experiment integrity. Re running that verdict on a timer adds no new signal, and a loop that accepts its own output to decide when to stop crosses into self acquittal ( acceptance gate.md ). Schedule the external wait that precedes it — experiments done → then audit once . See [ shared references/external cadence.md ](../shared references/external cadence.md). Audit experiment integrity for: $ARGUMENTS Why This Exists LLM agents can produce fraudulent experimental results through: 1. Fake ground truth — creating synthetic "reference" from model outputs, then reporting high agreement as performance 2. Score normalization — dividing metrics by the model's own max to get 0.99+ 3. Phantom results — claiming numbers from files that don't exist or functions never called 4. Insufficient scope — reporting 2 scene pilots as "comprehensive evaluation" These are NOT intentional deception — they are failure modes of optimizing agents that lack integrity constraints. This skill adds that constraint. Core Principle The executor collects file paths. The external reviewer backend reads code and judges integrity. The executor does NOT participate in integrity judgment. This follows shared references/reviewer independence.md and shared references/experiment integrity.md . Constants REVIEWER BACKEND = codex — Default: Codex MCP (ultra). Override with — reviewer: oracle pro for Oracle MCP, or — reviewer: manual for Manual Review MCP. If manual review MCP is unavailable, stop and print the install command; do not fall back to Codex. See shared references/reviewer routing.md . Reviewer Calling Convention When calling the reviewer, branch on REVIEWER BACKEND: If REVIEWER BACKEND = codex : Use mcp codex codex for new review threads. Use mcp codex codex reply for follow up rounds (reuse threadId). If REVIEWER BACKEND = manual : Use mcp manual review review for new review threads with: prompt: [exact same prompt that would go to Codex] config: {"model reasoning effort": "xhigh", "executor model": "<actual executor model ", "require reviewer model": true} Save the returned threadId . Use mcp manual review review reply for follow up rounds with: threadId: [saved manual review threadId] prompt: [follow up prompt] config: {"model reasoning effort": "xhigh", "executor model": "<actual executor model ", "require reviewer model": true} Prompt fidelity: the manual prompt must be exactly the same text that Codex would receive. Review tracing applies equally to both backends. Workflow Step 1: Collect Artifacts (Executor — Claude) Locate and list these files WITHOUT reading or summarizing their content: A verdict bearing manual response MUST begin with Reviewer Model: <exact model id — pass the model THIS session is actually running as in executor model . Missing, unknown, or same family identity cannot acquit; emit REVIEW UNAVAILABLE rather than guessing. If the executor model cannot be named, manual review's cross family claim is unprovable — say so in the report instead of asserting it. DO NOT summarize, interpret, or explain any file content. Only collect paths. Step 2: Send to Reviewer Based on the selected reviewer backend (see Reviewer Calling Convention), pass ONLY file paths and the audit checklist to the reviewer. The reviewer reads everything directly. For codex , call mcp codex codex with: model: gpt 6 astra config: {"model reasoning effort": "ultra"} sandbox: read only cwd: [project directory] prompt: [the exact full prompt below] For manual , call mcp manual review review with: config: {"model reasoning effort": "xhigh", "executor model": "<actual executor model ", "require reviewer model": true} prompt: [the exact full prompt below] Manual review cannot use Codex only model , sandbox , or cwd ; include the same file paths in the prompt so the user can inspect them. Use this exact prompt for both backends: Step 3: Parse and Write Report (Executor — Claude) Parse the reviewer's response and write EXPERIMENT AUDIT.md : Also write EXPERIMENT AUDIT.json for machine consumption: Step 4: Print Summary Integration with Other Skills Automatic in /research pipeline (advisory, never blocks) When integrated into the pipeline, this skill runs automatically after /experiment bridge and before /auto review loop : Never blocks the pipeline. Even on FAIL, the pipeline continues — but claims carry visible integrity tags. Read by /result to claim (if exists) Read by /paper write (if exists) Key Rules Reviewer independence : executor collects paths, reviewer judges. Period. Never block : warn loudly, never halt the pipeline. File as switch : no EXPERIMENT AUDIT.md = skill was never run = zero impact on existing behavior. Cross model : the reviewer MUST be a different model family from the executor. Honest about limits : the audit catches common patterns, not all possible fraud. It is a safety net, not a guarantee. Acknowledgements Motivated by community reported integrity issues ( 57, 131) where executor agents created fake ground truth and self normalized scores. Review Tracing After each reviewer call ( mcp codex codex , mcp codex codex reply , mcp manual review review , or mcp manual review review reply ), save the trace following shared references/review tracing.md (Policy C — forensic; never silently skip). Use save trace.sh (resolved per the chain in shared references/integration contract.md §2) or write files directly to .aris/traces/<skill /<date run<NN / . Respect the trace: parameter (default: full ).

experiment-audit