citation-audit
Zero-context verification that every bibliographic entry in the paper is real, correctly attributed, and used in a context the cited paper actually supports — catching hallucinated authors, wrong years, fabricated venues, version mismatches, and wrong-context citations. Use when user says "审查引用", "c
By wanshuiyin · 370 installs
npx skills add wanshuiyin/auto-claude-code-research-in-sleep --skill citation-audit
Source repository · Upstream listing
Citation Audit
🔒 Do not wrap this skill in /loop , /schedule , or CronCreate . It is
verdict bearing — it judges bibliographic correctness. Re running that verdict
on a timer adds no new signal (it changes only when the bibliography
changes). Schedule the external wait that precedes it — bibliography
finalized → then audit once . See
[ shared references/external cadence.md ](../shared references/external cadence.md).
Verify every \cite{...} in a paper against three independent layers:
1. Existence — the cited paper actually exists at the claimed arXiv ID / DOI / venue.
2. Metadata correctness — author names, year, venue, and title match canonical sources (DBLP, arXiv, ACL Anthology, Nature, OpenReview, etc.).
3. Context appropriateness — the cited paper actually supports the claim it is being used to support in the manuscript.
This skill is the fourth layer of \aris{}'s evidence and claim assurance, complementing experiment audit (code), result to claim (science verdict), and paper claim audit (numerical claims). Together they form a bottom up integrity stack from raw evaluation code to manuscript bibliography.
When to Use This Skill
Run before submission. The right gating point is:
After paper write has produced the LaTeX draft and bib file
After paper claim audit has verified numerical claims
Before final paper compile for submission
Do not run this on a half written draft — most of the work is in cross checking each \cite against context, which is wasted on placeholder text.
What This Skill Catches
The dangerous citation problems are not wildly fake citations — those are easy to spot. The dangerous ones are:
Wrong context citations : real paper, but the cited claim is not what that paper actually establishes (e.g., citing Self Refine to support "self feedback produces correlated errors" — Self Refine actually argues the opposite).
Author hallucinations : anonymous author placeholders that slipped through, missing co authors, wrong order.
Title drift : arXiv v1 vs v3 with different titles silently merged.
Venue confusion : arXiv preprint cited but the official venue is now CVPR/ICML/NeurIPS — using the wrong record.
Year mismatch : arXiv 2023 preprint with 2024 conference acceptance, year reported inconsistently.
Phantom DOIs : DOI looks real but does not resolve.
Self citation drift : your own prior work cited with year off by one.
Constants
REVIEWER MODEL = gpt 6 astra — Used via Codex MCP. Default for cross model review with web access.
CONTEXT POLICY = fresh — Each audit run uses a new reviewer thread (REVIEWER BIAS GUARD). Never codex reply .
WEB SEARCH = required — The reviewer must perform real web/DBLP/arXiv lookups, not pattern match from memory.
OUTPUT = CITATION AUDIT.md — Human readable per entry verdict report.
STATE = CITATION AUDIT.json — Machine readable verdict ledger consumable by downstream tools.
SOFT ONLY = false — When true (set via — soft only / — soft only flag), the audit runs all three layers normally but forbids any .bib file mutation . Findings that would otherwise mutate the bib (FIX / REPLACE / REMOVE) are translated into per occurrence sentence rewrite proposals against the citing .tex files. Used by /resubmit pipeline Phase 1 to honor the user's hard "freeze the bib" constraint.
RENDER HTML = true — When true (default), auto render CITATION AUDIT.md to HTML after writing the report. Uses full Codex review gate (audit class artifact — render fidelity check matches the skill's cross model audit invariant). Set false to skip, or pass — render html: false .
Workflow
Step 1: Discover bib file and section files
Locate:
references.bib (or paper.bib / similar) under the paper directory
All .tex files containing \cite{...} calls (typically sec/ or sections/ )
If multiple bib files exist, audit each separately.
Step 2: Extract all (cite key, context) pairs
For each \cite{key1,key2,...} invocation in the paper:
Record the cite key
Record the file + line number
Record the surrounding sentence (≥ 1 full sentence around the cite, for context check)
Output a flat list of (key, file, line, surrounding sentence) tuples.
Also build the inverse: for each bib entry, the list of all places it is cited.
Define two protocol sets used throughout the rest of the workflow: cited keys is the set of unique cite keys appearing in any \cite{...} invocation across the audited .tex files (de duplicated), and bib keys is the set of keys parsed from the audited bib file(s). cited keys drives Step 3 (audit only cited entries); bib keys \ cited keys is the uncited residual surfaced by the uncited opt in.
If the user passed uncited , also compute the set difference bib keys \ cited keys here and stash it for use in Steps 5 and the JSON aggregation; see "Uncited Entry Detection (opt in)" below for the protocol. The set diff is a string operation only and does not consume reviewer budget.
Save the extracted contexts to paper/.aris/citation audit/contexts.txt so the reviewer can read it directly. Use the paper dir relative path .aris/citation audit/contexts.txt when recording the file in audited input hashes ; do not stage under /tmp or other transient locations that the verifier cannot rehash later.
Step 3: Send each entry to fresh cross model reviewer
For each cited bib entry — i.e., each key in cited keys with at least one extracted citation context — invoke mcp codex codex (NOT codex reply — fresh thread per entry, or batch with explicit per entry isolation). Do not send entries in bib keys \ cited keys to the reviewer; those are detect only and surface only when uncited is explicitly enabled (see "Uncited Entry Detection" below).
Save the response to .aris/traces/citation audit/<date runNN/<key .md per the review tracing protocol.
Step 4: Aggregate verdicts
Build CITATION AUDIT.json following the schema defined in "Submission
Artifact Emission" below (single authoritative schema for this file).
Per entry ledger data goes under details.per entry , not under a
top level entries field. The top level verdict is a single overall
value (PASS / WARN / FAIL / NOT APPLICABLE / BLOCKED / ERROR) derived
from per entry verdicts per the decision table in "Submission Artifact
Emission"; the top level summary is a one line human readable string.
Concretely, details carries the per entry ledger:
See "Submission Artifact Emission" for the full artifact (top level
fields audit skill , verdict , reason code , summary ,
audited input hashes , trace path , thread id , reviewer model ,
reviewer reasoning , generated at , details ).
Step 5: Generate human readable report
Write CITATION AUDIT.md :
When uncited is set, append the following section after "All Clean Entries":
Step 6: Apply fixes (interactive)
For each FIX/REPLACE/REMOVE verdict, prompt the user:
If AUTO APPLY = true , apply all FIX level changes (metadata corrections only). REPLACE and REMOVE always require human approval — they involve content changes.
Step 7: Recompile and verify
Confirm:
No new Citation undefined warnings
No Reference undefined warnings
Page count unchanged or only minimally affected by metadata fixes
Uncited Entry Detection (opt in)
Default : disabled. Existing users see no behavior change — only \cite{...} keys are audited, and bib entries with no \cite reference in the manuscript are silently ignored.
Opt in : pass uncited on invocation. The skill then performs a set diff after Step 2 and reports bib entries that appear in any audited bib file(s) but are not cited anywhere in the paper. Detect only — uncited entries are not sent to the cross model reviewer, so there is no extra reviewer/web lookup cost.
Why opt in
This skill's headline output is the three axis audit on cited entries. Surfacing uncited bib entries by default would (a) change long form output for every existing run, and (b) noise up the verdict for users who intentionally maintain a superset bib file (e.g., shared lab bib, in progress section reorder where the cite has been removed but the entry intentionally retained). The flag preserves zero behavior change for existing callers.
Effect when enabled
When uncited is set:
CITATION AUDIT.md gains a Uncited Entries (opt in) section listing the keys with a one line suggestion each: prune (entry is dead weight; recommend deleting) or check (entry might be intentional; flag for user review). Default suggestion is prune ; only emit check when there is concrete local evidence (e.g., a TODO comment in a .tex file mentioning the key, or a recently removed \cite visible in git diff ). Do not infer intent from the bib key string alone.
CITATION AUDIT.json details gains an uncited entries array; see "Submission Artifact Emission" below for the schema.
The top level verdict is unchanged : uncited entries do not upgrade or downgrade the PASS / WARN / FAIL / etc. classification. The reason code and summary are likewise unchanged in shape; only the details.uncited entries field appears.
Verifier gates and downstream skills ( paper writing Phase 6, verify paper audits.sh ) MUST NOT treat the presence of uncited entries as a blocking signal.
When opt in is appropriate
Pre submission cleanup (drop dead bib entries before sharing camera ready ZIP).
Shared lab bib file where the paper uses a subset and the user wants to confirm what is in scope.
Recurring audits where the user has previously seen the uncited count and wants to track whether it changed.
Fallback when bib enumeration fails
If uncited is enabled but full bib key enumeration fails (e.g., malformed bib syntax that the parser cannot recover), the cited entry audit must still proceed if at all possible. In that case:
Do not alter the top level verdict , reason code , or summary .
Emit details.uncited entries as an empty array [] .
Add details.uncited entries status: "unavailable" plus a one line note explaining why (e.g., "bib parser could not enumerate keys; cited entry audit completed normally" ).
Verifier gates and downstream skills MUST treat unavailable the same as the field being absent: not blocking.
If the bib file cannot be read well enough to audit even the cited entries, fall back to the existing BLOCKED / bib unreadable path defined in the verdict decision table; this is the same behavior as the no flag default.
Key Rules
Fresh reviewer thread per audit run — never reuse prior review context
Web access required — the reviewer must do real lookups, not memory pattern match
Wrong context metadata — a real paper used to support a wrong claim is more dangerous than a typo in author name
REPLACE/REMOVE require human approval — never auto modify content claims
Always emit, never block — this skill always writes CITATION AUDIT.json with a verdict; the decision to block finalization lives in paper writing Phase 6 + verify paper audits.sh , driven by the assurance level. See "Submission Artifact Emission" below.
Run once per submission — the audit is wall clock expensive (web lookups for each entry); not for every save
Uncited detection is opt in only — never auto enable; never block on uncited entries; existing callers must observe identical output if they do not pass uncited
Under soft only , citation audit emits text rewrite proposals only; bib files are never mutated regardless of finding severity. The audit semantics (existence + metadata + context) and the per entry KEEP/FIX/REPLACE/REMOVE ledger are preserved verbatim; only the action layer is translated to per occurrence sentence rewrites in the citing .tex files. Refuse any downstream proposed bib edit while soft only is s