research-refine
Turn a vague research direction into a problem-anchored, elegant, frontier-aware, implementation-oriented method plan via iterative GPT-6-Astra review. Use when the user says "refine my approach", "帮我细化方案", "decompose this problem", "打磨idea", "refine research plan", "细化研究方案", or wants a concrete res
By wanshuiyin · 429 installs
npx skills add wanshuiyin/auto-claude-code-research-in-sleep --skill research-refine
Source repository · Upstream listing
Research Refine: Problem Anchored, Elegant, Frontier Aware Plan Refinement
Refine and concretize: $ARGUMENTS
Overview
Use this skill when the research problem is already visible but the technical route is still fuzzy. The goal is not to produce a bloated proposal or a benchmark shopping list. The goal is to turn a vague direction into a problem focused method minimal validation document that is concrete enough to implement, elegant enough to feel paper worthy, and current enough to resonate in the foundation model era.
Four principles dominate this skill:
1. Do not lose the original problem. Freeze an immutable Problem Anchor and reuse it in every round.
2. The smallest adequate mechanism wins. Prefer the minimal intervention that directly fixes the bottleneck.
3. One paper, one dominant contribution. Prefer one sharp thesis plus at most one supporting contribution.
4. Modern leverage is a prior, not a decoration. When LLM / VLM / Diffusion / RL / distillation / inference time scaling naturally fit the bottleneck, use them concretely. Do not bolt them on as buzzwords.
Constants
REVIEWER MODEL = gpt 6 astra — Reviewer model used via Codex MCP.
MAX ROUNDS = 5 — Maximum review revise rounds.
SCORE THRESHOLD = 9 — Minimum overall score to stop.
OUTPUT DIR = refine logs/ — Directory for round files and final report.
MAX LOCAL PAPERS = 15 — Maximum local papers/notes to scan for grounding.
MAX CORE EXPERIMENTS = 3 — Default cap for core validation blocks inside this skill.
MAX PRIMARY CLAIMS = 2 — Soft cap for paper level claims. Prefer one dominant claim plus one supporting claim.
MAX NEW TRAINABLE COMPONENTS = 2 — Soft cap for genuinely new trainable pieces. Exceed only if the paper breaks otherwise.
Override via argument if needed, e.g. /research refine "problem approach" max rounds: 3, threshold: 9 .
State Persistence (Checkpoint Recovery)
Long running refinement sessions may fail mid way (e.g., API timeout, context compaction, or session interruption). To avoid losing completed work, persist state to refine logs/REFINE STATE.json after each phase boundary:
Field definitions:
Field Values Meaning
phase "anchor" / "proposal" / "review" / "refine" / "done" Last completed phase
round 0–MAX ROUNDS Current round number
threadId string or null Reviewer thread ID for codex reply continuity
last score number or null Most recent overall score from reviewer
last verdict string or null Most recent verdict (READY / REVISE / RETHINK)
status "in progress" / "completed" Loop status
timestamp ISO 8601 When state was last written
Write rules:
Write after each phase completes (not before). Overwrite each time — only the latest state matters.
On completion (Phase 5 finished), set "status": "completed" .
Output Structure
Every round N refinement.md must contain a full anchored proposal , not just incremental fixes.
Workflow
Initialization (Checkpoint Recovery)
Before starting any phase, check whether a previous run left a checkpoint:
1. Check for refine logs/REFINE STATE.json :
If it does not exist → fresh start (proceed to Phase 0 normally)
If it exists AND status is "completed" → fresh start (delete state file, previous run finished)
If it exists AND status is "in progress" AND timestamp is older than 24 hours → fresh start (stale state from a killed/abandoned run — delete the file)
If it exists AND status is "in progress" AND timestamp is within 24 hours → resume
2. On resume , read the state file and recover context:
Read all existing refine logs/round .md files to restore prior work
Read refine logs/score history.md if it exists
Recover threadId for reviewer thread continuity
Log to the user: "Checkpoint found. Resuming after phase: {phase}, round: {round}."
Jump to the next phase based on the saved phase value:
Saved phase What was completed Resume from
"anchor" Phase 0 done Phase 1 (read anchor from round 0 context)
"proposal" Phase 1 done Phase 2 (read round 0 initial proposal.md )
"review" Phase 2 or 4 done Phase 3 (read latest round N review.md )
"refine" Phase 3 done Phase 4 (read latest round N refinement.md )
3. On fresh start , ensure refine logs/ directory exists and proceed to Phase 0.
Phase 0: Freeze the Problem Anchor
Before proposing anything, extract the user's immutable bottom line problem. This anchor must be copied verbatim into every proposal and every refinement round.
Write:
Bottom line problem : What technical problem must be solved?
Must solve bottleneck : What specific weakness in current methods is unacceptable?
Non goals : What is explicitly not the goal of this project?
Constraints : Compute, data, time, tooling, venue, deployment limits.
Success condition : What evidence would make the user say "yes, this method addresses the actual problem"?
If later reviewer feedback would change the problem being solved, mark that as drift and push back or adapt carefully.
Checkpoint: Write refine logs/REFINE STATE.json with {"phase": "anchor", "round": 0, "threadId": null, "last score": null, "last verdict": null, "status": "in progress", "timestamp": "<now "} .
Phase 1: Build the Initial Proposal
Step 1.1: Scan Grounding Material
Check papers/ and literature/ first. Read only the relevant parts needed to answer:
What mechanism do current methods use?
Where exactly do they fail for this problem?
Which recent LLM / VLM / Diffusion / RL era techniques are actually relevant here?
What training objectives, representations, or interfaces are reusable?
What details distinguish a real method from a renamed high level idea?
If local material is insufficient, search recent top venue/arXiv work online. Focus on method sections, training setup, and failure modes , not just abstracts.
Step 1.2: Identify the Technical Gap
Do not stop at generic research questions. Make the gap operational:
1. Current pipeline failure point : where does the baseline break?
2. Why naive fixes are insufficient : larger context, more data, prompting, memory bank, or stacking more modules.
3. Smallest adequate intervention : what is the least additional mechanism that could plausibly fix the bottleneck?
4. Frontier native alternative : is there a more current route using foundation model era primitives that better matches the bottleneck?
5. Core technical claim : what exact mechanism claim could survive top venue scrutiny?
6. Required evidence : what minimum proof is needed to defend that claim?
Step 1.3: Choose the Sharpest Route
Before locking the method, compare two candidate routes if both are plausible:
Route A: Elegant minimal route — the smallest mechanism that directly targets the bottleneck.
Route B: Frontier native route — a more modern route that uses LLM / VLM / Diffusion / RL / distillation / inference time scaling only if it gives a cleaner or stronger story.
Then decide:
Which route is more likely to become a strong paper under the stated constraints?
Which route has the cleaner novelty story relative to the closest work?
Which route avoids contribution sprawl?
If both routes are weak, rethink the framing instead of combining them into a larger system by default.
Step 1.4: Concretize the Method First
The proposal must answer "how would we actually build this?" Prefer method detail over broad experimentation and prefer reuse over invention.
Cover:
1. One sentence method thesis : the single strongest mechanism claim.
2. Contribution focus : one dominant contribution and at most one supporting contribution.
3. Complexity budget : what is frozen or reused, what is new, and what tempting additions are intentionally excluded.
4. System graph : modules, data flow, inputs, outputs.
5. Representation design : what latent, embedding, plan token, reward signal, memory state, or alignment space is used?
6. Training recipe : data source, supervision, pseudo labeling, negatives, curriculum, losses, weighting, stagewise vs joint training.
7. Inference path : how the trained components are used at test time and what signals flow where.
8. Why the mechanism stays small : why a larger stack is unnecessary.
9. Exact role of any frontier primitive : if you use an LLM / VLM / Diffusion / RL component, specify whether it acts as planner, teacher, critic, reward model, generator prior, search controller, or distillation source.
10. Failure handling : what could go wrong and what fallback or diagnostic exists?
11. Novelty and elegance argument : why this is more than naming a module and why the paper still looks focused.
If the method is still only described as "add a module" or "use a planner," it is not concrete enough.
Step 1.5: Design Minimal Claim Driven Validation
Experiments exist to validate the method, not to dominate the document.
For each core claim, define the smallest strong experiment that can validate it:
the claim being tested
the necessary baseline or ablation
the decisive metric
the expected directional outcome
Additional rules:
Ensure one experiment block directly supports the Problem Anchor .
If complexity risk exists, include one simplification or deletion check .
If a frontier primitive is central, include one necessity check showing why that choice matters.
Default to 1 3 core experiment blocks and leave the full execution roadmap to /experiment plan .
Step 1.6: Write the Initial Proposal
Save to refine logs/round 0 initial proposal.md .
Use this structure:
Checkpoint: Update refine logs/REFINE STATE.json with {"phase": "proposal", "round": 0, ...} .
Phase 2: External Method Review (Round 1)
Send the proposal to GPT 6 Astra for an elegance first, frontier aware,
method first review. The reviewer should spend most of the critique budget
on the method itself, not on expanding the experiment menu. For Codex MCP, do
not inline the whole rubric + proposal once the prompt becomes large. Instead,
write refine logs/codex round 1 review bundle.md containing the instructions
below plus the absolute path to refine logs/round 0 initial proposal.md , then
keep the MCP prompt short:
Bundle contents:
CRITICAL: Save the threadId from this call for all later rounds.
CRITICAL: Save the FULL raw response verbatim.
Save review to refine logs/round 1 review.md with the raw response in a <details block.
Checkpoint: Update refine logs/REFINE STATE.json with {"phase": "review", "round": 1, "threadId": "<saved ", "last score": <parsed , "last verdict": "<parsed ", ...} .
Phase 3: Parse Feedback and Revise the Method
Step 3.1: Parse the Review
Extract:
Problem Fidelity
Method Specificity
Contribution Quality
Frontier Leverage
Feasibility
Validation Focus
Venue Readiness
Overall score
Verdict
Drift Warning
Simplification Opportunities
Modernization Opportunities
Action items ranked by priority
Update refine logs/score history.md :
STOP CONDITION : If overall score = SCORE THRESHOLD, verdict is READY, and there is no unresolved drift warning, skip to Phase 5.
Step 3.2: Revise With an Anchor Check and a Simplicity Check
Before changing anything:
1. Copy the Problem Anchor verbatim .
2. Write an Anchor Check :
What is the original bottleneck?
Does the current method still solve it?
Which reviewer suggestions would cause drift if followed blindly?
3. Write a Simplicity Check :
What is the dominant contribution now?
What components can be removed, merged, or kept frozen?
Which reviewer suggestions add unnecessary complexity?
If a frontier primitive