ai-research-reproduction
Rigor Reproduce compatible skill slug for README-first deep learning repository reproduction. Use when the user wants an end-to-end, minimal-trustworthy flow that reads the repository first, selects the smallest documented inference or evaluation target, coordinates intake, setup, trusted execution,
By lllllllama · 310,958 installs
npx skills add lllllllama/rigorpilot-skills --skill ai-research-reproduction
Source repository · Upstream listing
ai research reproduction
Purpose
Guide README first deep learning reproduction toward a minimal trustworthy run
with auditable evidence. Reproduction is not "make it run by changing
anything"; faithfully read the README, environment, weights, datasets, and
documented commands, then record results and deviations. Start with
references/agent operating principles.md ; load
references/research rigor principles.md and
references/deep learning experiment principles.md when scientific meaning or
experiment details are at stake.
For first use problems, run scripts/doctor.py with the intended Python (read only; optional repo and require module ).
The deterministic entrypoint is scripts/orchestrate repro.py with a self contained bundled/ runtime, so this skill works when installed alone;
separately installed companion skills remain optional reusable entrypoints.
Use the entrypoint and help for routine runs; inspect its implementation when a concrete blocker or safety question requires it.
Executed commands persist lifecycle state, append only events, and full streamed
stdout/stderr under repro outputs/ runtime/<run id / . A CANCEL file in the
active run directory requests process tree cancellation.
For recovery, queues or model gates, read references/runtime and model adapter.md ; for the optional model/tool loop, read references/agent runner.md and use scripts/run agent.py .
Fit
Use this skill when all are true:
The target is an AI code repository with a README, scripts, configs, or
documented commands.
The request spans multiple trusted phases such as intake, setup, execution,
training verification, analysis, paper gap resolution, and reporting.
The desired result is a small reproducible target, not broad experimentation.
Do not use this skill for paper summaries, generic environment setup, isolated
repo scanning, standalone command execution, open ended research design, or
explicit candidate only exploration.
Trusted Target Selection
Choose the smallest target that can honestly demonstrate repository grounded
reproduction:
1. documented inference
2. documented evaluation
3. documented training startup or partial verification
4. full training only after explicit user confirmation
Treat README guidance as the primary reproduction intent. Use repository files
to clarify the README, not to silently replace it. When the README and paper
conflict, record the conflict and use paper context resolver only for the
narrow reproduction critical gap.
Workflow
1. Read the README and nearby repo signals.
2. Run the bundled repo intake and plan stage to extract commands and targets.
3. Select and justify the minimum trustworthy target.
4. Run env and assets bootstrap only for target specific environment,
checkpoint, dataset, and cache assumptions.
5. Run analyze project only when structure, insertion points, or suspicious
implementation patterns need read only clarification.
6. Use minimal run and audit for documented inference, evaluation, smoke, or sanity execution. Keep direct execution as the default; native shell syntax requires explicit review and authorization.
7. Use run train instead when the selected trusted target is training startup, short run verification, full kickoff, or resume.
8. Pause for human review before fuller training claims or any change that could
alter dataset, split, checkpoint, preprocessing, metric, loss, model
semantics, or result interpretation.
9. Award result match only when explicit expected metrics are compared under a recorded tolerance; observed metrics alone prove execution, not reproduction. Then write the standardized outputs and a concise final note in the user's language when practical.
10. Once the requested target and evidence checks are complete, return the bounded result and stop. Optional stages and further README commands are not automatic follow up work.
Patch Boundary
Prefer no repository edits. If edits are needed, keep them conservative and
auditable:
Try command line arguments, environment variables, path fixes, dependency
version fixes, or dependency file fixes before code changes.
Reproduction fixes are allowed when needed, but they must not be hidden. State
what changed, why it was necessary, whether it changes scientific meaning,
and whether it affects comparability with the paper, README, or baseline.
Avoid changing model architecture, core inference semantics, training logic,
loss functions, or experiment meaning.
If repository files must change, create a branch named
repro/YYYY MM DD short task , keep verified patch commits sparse, and record
README fidelity impact in PATCHES.md .
See references/patch policy.md .
Outputs
Always target repro outputs/ :
Use the templates under assets/ and the field rules in references/output spec.md .
Put the shortest high value summary in SUMMARY.md .
Put copyable commands in COMMANDS.md .
Put process evidence, assumptions, failures, and decisions in LOG.md .
Put scientific meaning and change effects in SCIENTIFIC CHANGELOG.md .
Put comparison anchors and protocol deviations in COMPARABILITY REPORT.md .
Put durable machine readable state in status.json .
Put branch, commit, validation, and README fidelity impact in PATCHES.md when needed.
Put the researcher's at a glance view in ANNOTATED README.md : the README replayed byte for byte—including its image, GIF, video, and HTML markup—with exactly one marked color annotation after every heading block. Never extract a text only surrogate. Generation must pass the built in strip/check round trip before the file is kept.
For original relative media/file context, use source adjacent readme to also write RIGORPILOT README.md beside the source README; inspect the reported path/status and never replace an unrelated existing file. See references/output spec.md .
Distinguish verified facts from inferred guesses.
Reference Loading
Load references/language policy.md when writing human readable outputs.
Load references/research rigor principles.md before making comparability, contribution, or research result claims.
Load references/deep learning experiment principles.md when dataset, split, metric, checkpoint, training, or evaluation details matter.
Consult ~/.rigorpilot/PERSONAL RIGOR.md if present, under references/continuous learning policy.md (advisory only; core wins).
Failed and later resolved runs are auto recorded as lessons via shared/scripts/lessons store.py ( RIGORPILOT LESSONS=0 disables).
Load references/research safety principles.md before protocol sensitive
decisions.
Load references/patch policy.md before modifying repository files.
Keep specialized logic in sub skills, scripts, templates, or references rather than expanding this entrypoint.