auto-review-loop
Autonomous multi-round research review loop. In Copilot CLI it defaults to the native complementary rubber-duck subagent with host-event model evidence; elsewhere it uses Codex, while explicit external reviewer overrides remain available. Implements fixes and re-reviews until a policy-approved posit
By wanshuiyin · 464 installs
npx skills add wanshuiyin/auto-claude-code-research-in-sleep --skill auto-review-loop
Source repository · Upstream listing
Auto Review Loop: Autonomous Research Improvement
🔒 Do not wrap this skill in /loop , /schedule , or CronCreate . It
already loops internally (review → fix → re review) and the reviewer carries
round to round memory in one threadId ( codex reply ). An external timer
re enters from the top each tick — fresh threadId , reviewer memory reset —
firing the verdict on wall clock time instead of on artifact change: zero new
signal, full token cost. If you want to schedule something, schedule the
external wait that precedes it (experiments done → then run this once). See
[ shared references/external cadence.md ](../shared references/external cadence.md).
Autonomously iterate: review → implement fixes → re review, until an independent reviewer gives a policy approved positive assessment or MAX ROUNDS is reached.
Context: $ARGUMENTS
Constants
MAX ROUNDS = 4
POSITIVE THRESHOLD: score = 6/10 AND verdict ∈ {"ready", "almost"} — both must hold. This matches the operative Phase E STOP CONDITION exactly; the verdict vocabulary is {"ready", "almost", "not ready"} (a high score with a "not ready" verdict does NOT stop the loop). Earlier wording here used or and a stale verdict set ("accept"/"sufficient"/"ready for submission") — that was an internal inconsistency; the AND form is authoritative.
REVIEW DOC: review stage/AUTO REVIEW.md (cumulative log) (fall back to ./AUTO REVIEW.md for legacy projects)
REVIEWER MODEL = gpt 6 astra — Default model for the Codex backend. Must be an OpenAI model (e.g., gpt 6 astra , gpt 6.1 sol ). Manual backend uses a model the user chooses — it must be a recognized model from a different family (OpenAI, Anthropic, Google, DeepSeek, Moonshot/Kimi, Qwen).
REVIEWER BACKEND — With no inline or project backend selection, start as auto ; Step 1 runs exactly one two call native marker/challenge probe for the first review. A bound Copilot CLI root session uses copilot native (built in complementary rubber duck subagent); an unbound/non Copilot host keeps the existing codex default. Inline or project selection of codex , antigravity / gemini , grok , claude code , oracle pro , agy , or manual bypasses the probe. Explicit — reviewer: copilot retains the compatibility copilot agent drive mode and its later Codex/manual finalizer. The native path gets both actual model IDs from host session events; it never needs COPILOT CLI or caller provided executor model . See shared references/reviewer routing.md .
OUTPUT DIR = review stage/ — All review stage outputs go here. Create the directory if it doesn't exist.
HUMAN CHECKPOINT = false — When true , pause after each round's review (Phase B) and present the score + weaknesses to the user. Wait for user input before proceeding to Phase C. The user can: approve the suggested fixes, provide custom modification instructions, skip specific fixes, or stop the loop early. When false (default), the loop runs fully autonomously.
COMPACT = false — When true , (1) read EXPERIMENT LOG.md and findings.md instead of parsing full logs on session recovery, (2) append key findings to findings.md after each round.
REVIEWER DIFFICULTY = medium — Controls how adversarial the reviewer is. Three levels:
medium (default): Current behavior — MCP based review, the executor controls what context the reviewer sees.
hard : Adds Reviewer Memory (the reviewer tracks its own suspicions across rounds) + Debate Protocol (the executor can rebut, the reviewer rules).
nightmare : Everything in hard + The selected reviewer reads the repo directly ( codex exec on the Codex route) (the executor cannot filter what the reviewer sees) + Adversarial Verification (the reviewer independently checks if code matches claims).
RENDER HTML = true — When true (default), auto render review stage/AUTO REVIEW.md to HTML on loop termination via /render html . Uses no review (the loop itself IS the cross model review; the HTML is a structural conversion). Set false to skip, or pass — render html: false .
⚠️ Nightmare + Manual incompatibility : If REVIEWER BACKEND = manual and REVIEWER DIFFICULTY = nightmare , STOP with:
"difficulty: nightmare requires direct repository access and is not compatible with reviewer: manual. Use difficulty: hard, or select a file reading reviewer."
💡 Override: /auto review loop "topic" — compact: true, human checkpoint: true, difficulty: hard
Reviewer Calling Convention
Resolve reviewer selection before any review: invocation — reviewer: →
project .aris/reviewer.json → host default. Use reviewer route.py with
host claude code tier regular (or actual Copilot host), resolved per
integration contract.md §2, and [reviewer backends.md](../shared references/reviewer backends.md).
Only the absence of both inline and project backend selection enters the
automatic host protocol below. Invalid configuration is REVIEW UNAVAILABLE .
After the host default is determined, resolve any project model/effort against
that backend; never carry model options into a different provider.
Save the resolved route, cwd and session id with REVIEW STATE.json ; restore
them on resume, including requested/reported model and actual effort. A changed
provider/model/fixed session option needs a new session. Legacy state without a
route follows the existing backend recovery below. Forward explicit reviewer
directives to child review skills. Codex pins/examples below apply to Codex.
When calling the reviewer, branch on REVIEWER BACKEND:
If REVIEWER BACKEND ∈ {antigravity, grok, claude code}:
Follow the selected first/reply protocol in reviewer backends.md , with the
same substantive prompt, artifact paths and memory. Set cwd to contain the
review inputs. Medium/hard use a new session then continuations; nightmare
starts a fresh artifact reading adversarial session on the selected route,
with the full Nightmare verification prompt below. It must inspect actual
code/results, not merely read the executor's report. Keep ordinary/nightmare
session ids distinct. Rebuttals use the session that made the contested review.
Check successful completion and required scope on every call. Reset completion,
scope and reported identity before dispatch; do not inherit them from a prior
round. Unavailable or incomplete review gives REVIEW UNAVAILABLE .
If neither an inline nor project reviewer backend was supplied:
Set REVIEWER BACKEND to auto . At Step 1 of the first round, resolve
copilot native evidence.py using the canonical four layer helper chain.
Generate a fresh binding <run id r<round review <8 random hex and invoke
marker , wait, then invoke challenge as two distinct root Bash calls .
Put the literal binding and concrete resolved helper path in both calls;
Copilot Bash calls do not share variables. If the challenge binds, set
REVIEWER BACKEND to copilot native and use that same challenge for the
first review. Do not issue a second activation challenge in Phase A. If it
exits 3 because no current Copilot root session is bound, use codex .
Explicit reviewer directives bypass this probe. If the helper is missing,
native acceptance is unavailable; use Codex only if that external backend
is positively available, otherwise emit REVIEW UNAVAILABLE .
If REVIEWER BACKEND = copilot native :
Read the challenge nonce and host reported executor model. Invoke the host's
native task tool with agent type: rubber duck ; do not start a subprocess
and do not specify a reviewer model. The prompt contains the exact standalone
ARIS REVIEW NONCE=<nonce line, artifact/diff paths, the output contract,
and (round 2+) review stage/REVIEWER MEMORY.md . It contains no executor
summary or fix narrative. After the task completes, invoke
copilot native evidence.py verify to create the evidence and raw response
artifacts. The verifier must observe one successful linked rubber duck
lifecycle and known, different host reported model families.
Pass the evidence to both review gate.py native evidence and
save trace.sh backend copilot native native evidence . A qualifying
native positive may stop directly; no external finalizer is needed. A native
negative continues with a fresh marker/challenge/subagent next round. Every
verdict bearing native call—including a hard mode rebuttal ruling—gets one
unique <run id, round, purpose artifact set and exactly one challenge.
Missing, same/unknown family, malformed, stale, or mismatched evidence is
never a verdict. If native complementary dispatch is unavailable, fall back
only to a positively available opposite family backend: Anthropic/Google
executor → Codex; OpenAI executor → manual with a reported non OpenAI model.
Otherwise emit REVIEW UNAVAILABLE . Full protocol:
shared references/reviewer routing.md .
If REVIEWER BACKEND = copilot :
Require executor model : if not provided → emit REVIEW UNAVAILABLE .
Determine executor family from executor model (see reviewer routing.md).
Router picks opposite family profile:
executor family=openai → profile="aris reviewer claude" (anthropic)
executor family=anthropic → profile="aris reviewer openai" (openai)
executor family=google → profile="aris reviewer openai" (openai, default cross)
executor family=unknown → REVIEW UNAVAILABLE (fail closed).
Verify the profile file exists at .github/agents/<profile .agent.md .
If missing → REVIEW UNAVAILABLE .
Read its model: field into REVIEWER MODEL , derive reviewer family
from that model string, and verify it differs from executor family . Pass
the same value through subprocess model ; never trust a caller supplied
family label or profile only pinning under an Auto session.
Identity assurance: executor model is caller declared routing input,
not runtime attestation. Record executor model source: caller declared , the
derived family relation , and independence verified: unverified . A pair of
different model strings must never be promoted to independently verified.
Capability gate: copilot help must advertise model , effort ,
and allow tool ; otherwise emit REVIEW UNAVAILABLE .
Use the copilot agent subprocess (documented Copilot CLI form)
with the selected profile, model "$REVIEWER MODEL" , effort xhigh ,
and allow tool=read for each review call.
Multi round: each round is a fresh copilot agent call with the same
profile; reviewer memory is carried via review stage/REVIEWER MEMORY.md artifact.
If copilot CLI is unavailable → REVIEW UNAVAILABLE for that drive round;
do not silently substitute another transport. A later positive Copilot
verdict still requires the separately documented Codex/manual finalizer.
See shared references/reviewer routing.md for the full copilot contract.
If REVIEWER BACKEND = codex :
Use mcp codex codex for new review threads.
Use mcp codex codex reply for follow up rounds (reuse threadId).
If REVIEWER BACKEND = manual :
Use mcp manual review review for new review threads with:
prompt: [exact same prompt that would go to Codex]
config: {"model reasoning effort": "xhigh", "executor model": "<actual executor model ", "require reviewer model": true}
Save the returned threadId .
Use mcp manual review review reply for follow up rounds with:
threadId: [saved manual review threadId]
prompt: [follow up prompt]
config: {"model reasoning effort": "xhigh", "executor model": "<actual executor model ", "require reviewer model": true}
A verdict bearing manual response MUST begin with
Reviewer Model: <exact model id . Derive reviewer family from that model
identity. Missing, unknown, or same family identity cannot acquit; for a
mandatory escalation, emit REVIEW UNAVAILABLE rather than guess