auto-review-loop

Autonomous multi-round research review loop. In Copilot CLI it defaults to the native complementary rubber-duck subagent with host-event model evidence; elsewhere it uses Codex, while explicit external reviewer overrides remain available. Implements fixes and re-reviews until a policy-approved posit

By wanshuiyin · 464 installs

npx skills add wanshuiyin/auto-claude-code-research-in-sleep --skill auto-review-loop

Source repository · Upstream listing

Auto Review Loop: Autonomous Research Improvement 🔒 Do not wrap this skill in /loop , /schedule , or CronCreate . It already loops internally (review → fix → re review) and the reviewer carries round to round memory in one threadId ( codex reply ). An external timer re enters from the top each tick — fresh threadId , reviewer memory reset — firing the verdict on wall clock time instead of on artifact change: zero new signal, full token cost. If you want to schedule something, schedule the external wait that precedes it (experiments done → then run this once). See [ shared references/external cadence.md ](../shared references/external cadence.md). Autonomously iterate: review → implement fixes → re review, until an independent reviewer gives a policy approved positive assessment or MAX ROUNDS is reached. Context: $ARGUMENTS Constants MAX ROUNDS = 4 POSITIVE THRESHOLD: score = 6/10 AND verdict ∈ {"ready", "almost"} — both must hold. This matches the operative Phase E STOP CONDITION exactly; the verdict vocabulary is {"ready", "almost", "not ready"} (a high score with a "not ready" verdict does NOT stop the loop). Earlier wording here used or and a stale verdict set ("accept"/"sufficient"/"ready for submission") — that was an internal inconsistency; the AND form is authoritative. REVIEW DOC: review stage/AUTO REVIEW.md (cumulative log) (fall back to ./AUTO REVIEW.md for legacy projects) REVIEWER MODEL = gpt 6 astra — Default model for the Codex backend. Must be an OpenAI model (e.g., gpt 6 astra , gpt 6.1 sol ). Manual backend uses a model the user chooses — it must be a recognized model from a different family (OpenAI, Anthropic, Google, DeepSeek, Moonshot/Kimi, Qwen). REVIEWER BACKEND — With no inline or project backend selection, start as auto ; Step 1 runs exactly one two call native marker/challenge probe for the first review. A bound Copilot CLI root session uses copilot native (built in complementary rubber duck subagent); an unbound/non Copilot host keeps the existing codex default. Inline or project selection of codex , antigravity / gemini , grok , claude code , oracle pro , agy , or manual bypasses the probe. Explicit — reviewer: copilot retains the compatibility copilot agent drive mode and its later Codex/manual finalizer. The native path gets both actual model IDs from host session events; it never needs COPILOT CLI or caller provided executor model . See shared references/reviewer routing.md . OUTPUT DIR = review stage/ — All review stage outputs go here. Create the directory if it doesn't exist. HUMAN CHECKPOINT = false — When true , pause after each round's review (Phase B) and present the score + weaknesses to the user. Wait for user input before proceeding to Phase C. The user can: approve the suggested fixes, provide custom modification instructions, skip specific fixes, or stop the loop early. When false (default), the loop runs fully autonomously. COMPACT = false — When true , (1) read EXPERIMENT LOG.md and findings.md instead of parsing full logs on session recovery, (2) append key findings to findings.md after each round. REVIEWER DIFFICULTY = medium — Controls how adversarial the reviewer is. Three levels: medium (default): Current behavior — MCP based review, the executor controls what context the reviewer sees. hard : Adds Reviewer Memory (the reviewer tracks its own suspicions across rounds) + Debate Protocol (the executor can rebut, the reviewer rules). nightmare : Everything in hard + The selected reviewer reads the repo directly ( codex exec on the Codex route) (the executor cannot filter what the reviewer sees) + Adversarial Verification (the reviewer independently checks if code matches claims). RENDER HTML = true — When true (default), auto render review stage/AUTO REVIEW.md to HTML on loop termination via /render html . Uses no review (the loop itself IS the cross model review; the HTML is a structural conversion). Set false to skip, or pass — render html: false . ⚠️ Nightmare + Manual incompatibility : If REVIEWER BACKEND = manual and REVIEWER DIFFICULTY = nightmare , STOP with: "difficulty: nightmare requires direct repository access and is not compatible with reviewer: manual. Use difficulty: hard, or select a file reading reviewer." 💡 Override: /auto review loop "topic" — compact: true, human checkpoint: true, difficulty: hard Reviewer Calling Convention Resolve reviewer selection before any review: invocation — reviewer: → project .aris/reviewer.json → host default. Use reviewer route.py with host claude code tier regular (or actual Copilot host), resolved per integration contract.md §2, and [reviewer backends.md](../shared references/reviewer backends.md). Only the absence of both inline and project backend selection enters the automatic host protocol below. Invalid configuration is REVIEW UNAVAILABLE . After the host default is determined, resolve any project model/effort against that backend; never carry model options into a different provider. Save the resolved route, cwd and session id with REVIEW STATE.json ; restore them on resume, including requested/reported model and actual effort. A changed provider/model/fixed session option needs a new session. Legacy state without a route follows the existing backend recovery below. Forward explicit reviewer directives to child review skills. Codex pins/examples below apply to Codex. When calling the reviewer, branch on REVIEWER BACKEND: If REVIEWER BACKEND ∈ {antigravity, grok, claude code}: Follow the selected first/reply protocol in reviewer backends.md , with the same substantive prompt, artifact paths and memory. Set cwd to contain the review inputs. Medium/hard use a new session then continuations; nightmare starts a fresh artifact reading adversarial session on the selected route, with the full Nightmare verification prompt below. It must inspect actual code/results, not merely read the executor's report. Keep ordinary/nightmare session ids distinct. Rebuttals use the session that made the contested review. Check successful completion and required scope on every call. Reset completion, scope and reported identity before dispatch; do not inherit them from a prior round. Unavailable or incomplete review gives REVIEW UNAVAILABLE . If neither an inline nor project reviewer backend was supplied: Set REVIEWER BACKEND to auto . At Step 1 of the first round, resolve copilot native evidence.py using the canonical four layer helper chain. Generate a fresh binding <run id r<round review <8 random hex and invoke marker , wait, then invoke challenge as two distinct root Bash calls . Put the literal binding and concrete resolved helper path in both calls; Copilot Bash calls do not share variables. If the challenge binds, set REVIEWER BACKEND to copilot native and use that same challenge for the first review. Do not issue a second activation challenge in Phase A. If it exits 3 because no current Copilot root session is bound, use codex . Explicit reviewer directives bypass this probe. If the helper is missing, native acceptance is unavailable; use Codex only if that external backend is positively available, otherwise emit REVIEW UNAVAILABLE . If REVIEWER BACKEND = copilot native : Read the challenge nonce and host reported executor model. Invoke the host's native task tool with agent type: rubber duck ; do not start a subprocess and do not specify a reviewer model. The prompt contains the exact standalone ARIS REVIEW NONCE=<nonce line, artifact/diff paths, the output contract, and (round 2+) review stage/REVIEWER MEMORY.md . It contains no executor summary or fix narrative. After the task completes, invoke copilot native evidence.py verify to create the evidence and raw response artifacts. The verifier must observe one successful linked rubber duck lifecycle and known, different host reported model families. Pass the evidence to both review gate.py native evidence and save trace.sh backend copilot native native evidence . A qualifying native positive may stop directly; no external finalizer is needed. A native negative continues with a fresh marker/challenge/subagent next round. Every verdict bearing native call—including a hard mode rebuttal ruling—gets one unique <run id, round, purpose artifact set and exactly one challenge. Missing, same/unknown family, malformed, stale, or mismatched evidence is never a verdict. If native complementary dispatch is unavailable, fall back only to a positively available opposite family backend: Anthropic/Google executor → Codex; OpenAI executor → manual with a reported non OpenAI model. Otherwise emit REVIEW UNAVAILABLE . Full protocol: shared references/reviewer routing.md . If REVIEWER BACKEND = copilot : Require executor model : if not provided → emit REVIEW UNAVAILABLE . Determine executor family from executor model (see reviewer routing.md). Router picks opposite family profile: executor family=openai → profile="aris reviewer claude" (anthropic) executor family=anthropic → profile="aris reviewer openai" (openai) executor family=google → profile="aris reviewer openai" (openai, default cross) executor family=unknown → REVIEW UNAVAILABLE (fail closed). Verify the profile file exists at .github/agents/<profile .agent.md . If missing → REVIEW UNAVAILABLE . Read its model: field into REVIEWER MODEL , derive reviewer family from that model string, and verify it differs from executor family . Pass the same value through subprocess model ; never trust a caller supplied family label or profile only pinning under an Auto session. Identity assurance: executor model is caller declared routing input, not runtime attestation. Record executor model source: caller declared , the derived family relation , and independence verified: unverified . A pair of different model strings must never be promoted to independently verified. Capability gate: copilot help must advertise model , effort , and allow tool ; otherwise emit REVIEW UNAVAILABLE . Use the copilot agent subprocess (documented Copilot CLI form) with the selected profile, model "$REVIEWER MODEL" , effort xhigh , and allow tool=read for each review call. Multi round: each round is a fresh copilot agent call with the same profile; reviewer memory is carried via review stage/REVIEWER MEMORY.md artifact. If copilot CLI is unavailable → REVIEW UNAVAILABLE for that drive round; do not silently substitute another transport. A later positive Copilot verdict still requires the separately documented Codex/manual finalizer. See shared references/reviewer routing.md for the full copilot contract. If REVIEWER BACKEND = codex : Use mcp codex codex for new review threads. Use mcp codex codex reply for follow up rounds (reuse threadId). If REVIEWER BACKEND = manual : Use mcp manual review review for new review threads with: prompt: [exact same prompt that would go to Codex] config: {"model reasoning effort": "xhigh", "executor model": "<actual executor model ", "require reviewer model": true} Save the returned threadId . Use mcp manual review review reply for follow up rounds with: threadId: [saved manual review threadId] prompt: [follow up prompt] config: {"model reasoning effort": "xhigh", "executor model": "<actual executor model ", "require reviewer model": true} A verdict bearing manual response MUST begin with Reviewer Model: <exact model id . Derive reviewer family from that model identity. Missing, unknown, or same family identity cannot acquit; for a mandatory escalation, emit REVIEW UNAVAILABLE rather than guess