academic-experiments

$npx mdskill add joshua-zyy/academic-paper-writer/academic-experiments

Audit and produce evidence inventory for CS/AI/ML papers experiments

  • Identifies and runs experiments to verify results for academic papers.
  • Deploys manifest.yaml for binding rules and axes.
  • Decides actions based on loaded material from stance.md, red-lines.md, output-contract.md, and anti-patterns.md.
  • Delivers evidence inventory with annotations and protocol risk assessments.

SKILL.md

.github/skills/academic-experimentsView on GitHub ↗
---
name: academic-experiments
description: "Audit, run, or verify experimental evidence for CS/AI/ML papers. Produces Evidence Inventory with evidence_type annotations (newly_run/preexisting_artifact/user_claim) and Protocol Risk assessments. Use when: checking if experiment results are reproducible, auditing existing experiment artifacts, running minimal reproducible commands, evaluating checkpoints without full retraining, documenting protocol risks like data leakage or missing baselines. Triggers on: 复核实验, run experiments, 实验结果, experiment evidence, verify results, 实验验证, evidence inventory, protocol risk, 跑实验, check results, reproduce experiments, 实验审计."
---

# Academic Experiments

将此 skill 视为"实验取证代理"，目标是建立最短且可信的证据链，而不是尽量多跑实验。

## Router Protocol

1. Read `manifest.yaml`. It declares `always_load` files, `axes`, and `references.on_demand`.
2. Read every file listed under `always_load`. These are the skill's binding rules — not reference material.
3. Apply the loaded material as constraints:
   - `stance.md` defines non-negotiable rules, evidence type semantics, failure degradation, and scope.
   - `red-lines.md` defines absolute prohibitions. Do not negotiate these.
   - `output-contract.md` defines deliverables per mode and claim-readiness classification.
   - `anti-patterns.md` defines known failure modes and their correct alternatives.
4. Detect the mode using the manifest's `mode` axis: `experiment-evidence-pass`, `evidence-inventory-only`, or `minimal-reproducible-run`. Align evidence type semantics to `../shared/core/evidence-policy.md`.
5. Echo the selected mode to the user before executing.
6. Reach for `references/` only when the manifest's `references.on_demand` condition is satisfied.

## Modes

| Mode | Use when |
|---|---|
| `experiment-evidence-pass` | Full audit: inventory + run + record + risk analysis |
| `evidence-inventory-only` | Inventory existing artifacts only, no execution |
| `minimal-reproducible-run` | Execute minimal reproducible command (e.g. eval existing checkpoint) |

## Agent Dispatch

`agents/experiment_agent.md` is dispatched by `academic-paper-writer` orchestrator at Step 4. The agent may run experiments but must not modify project source code or data files, nor write paper prose independently.

## Independent Use

| Input | Mode | Priority | Behavior |
|---|---|---|---|
| `repo_path` + no run mode | experiment-evidence-pass | 2 (path trigger) | Full audit: inventory → env → minimal run → risk |
| `repo_path` + "inspect only" | evidence-inventory-only | 1 (explicit) | Inventory only, no commands |
| `repo_path` + specific command | minimal-reproducible-run | 1 (explicit) | Verify env → execute → record |
| No `repo_path` | — | 3 (no input) | Ask path, or auto-detect entry files |

| Scenario | Recommended |
|---|---|
| Just auditing/reproducing evidence | This skill (standalone) |
| Writing results into paper prose | academic-paper-writer orchestrator |
| Draft results need verification | This skill → academic-reviser |

More from joshua-zyy/academic-paper-writer

SkillDescription
academic-citationSearch, verify, and map citations for CS/AI/ML papers. Produces VERIFIED/UNVERIFIED reference lists with Citation-to-Claim maps and Exemplar Sets. Use when: finding references for a paper section, verifying citation accuracy, building exemplar sets for introduction/related work learning, checking if existing citations are real and accurate, supplementing local literature library. Triggers on: 找引用, 文献检索, citation pass, find references, reference check, 补文献, citation verification, Exemplar Set, search papers, verify citation, 核验文献, 查引用, literature search, reference verification, citation verification with reading, 引用确认, 全文阅读验证.
academic-figureCreate, revise, or audit academic data/result figures for CS/AI/ML papers. Data/result plots default to Python-generated editable SVG with CS/AI/ML-specific design rules for benchmarks, ablations, training dynamics, robustness, diagnostics, distributions, confusion matrices, and efficiency tradeoffs. Use when: generating plots from experiment results or numeric data, auditing publication figures, suggesting data-driven figure types, revising chart colors/layouts/labels, or preparing figure QA reports. Model framework diagrams, architecture diagrams, overview diagrams, and complex mechanism schematics are outside this skill's automatic drawing scope; provide only manual figure requirements or caption/blueprint notes when needed. Triggers on: 绘图, figure, chart, 画图, 实验图, 训练曲线, 消融实验, 对比图, 混淆矩阵, 结果图, 性能图, 鲁棒性图, 效率图, plot, publication figure, 数据可视化, generate plot, figure blueprint, 建议图表类型, figure audit, 审查图表, figure revision, 修改图表.
academic-latex-layoutUse when working on academic paper LaTeX layout, figure/table float placement, draft-to-template LaTeX generation, existing .tex project layout repair, figures piling up, figures far from text, large blank areas, two-column figure/table placement, page-limit layout cleanup, or compiled PDF layout review.
academic-paper-writerUse when writing or revising CS/AI/ML papers from research notes, code repositories, local literature libraries, target venue requirements, or section-level drafting requests. Triggers include 写论文, 初稿, paper draft, write introduction, write method, full paper outline, section-by-section drafting, 证据闭环, 论文起草, 从零写论文, 逐节写作.
academic-polishingPolish academic prose, de-AI-ify text, control claim strength, or rewrite method sections for CS/AI/ML papers. Executes Prose Quality Gate, Claim Strength Audit, and de-AI pass. Use when: removing AI writing patterns from paper text, adjusting claim strength to match evidence level, rewriting method sections with proper narrative flow, improving academic writing quality, checking for overclaiming. Triggers on: 润色, polish, improve writing, 去AI, de-AI, claim strength, 改写, rewrite method, prose quality, 降级表述, remove AI patterns, academic writing polish, 学术润色, 去AI化, 降级结论, improve prose.
academic-reviserSelf-review, audit, or verify CS/AI/ML paper drafts as a critical peer reviewer. Three-round review (evidence→argument→style) with Verification Status and debt tracking. Use when: reviewing a paper draft before submission, checking evidence compliance of claims, simulating peer reviewer feedback, verifying citation closure and evidence debts, performing cross-section consistency checks. Triggers on: self review, 自查, verification, 审稿, evidence compliance, peer review, 论文审查, draft audit, 验证论文, 检查引用, cross-section review, 审修, draft verification.
academic-venue-researchResearch target venue requirements and writing style for CS/AI/ML papers. Produces venue-brief.md with submission requirements and detailed writing style analysis. Use when: researching target journal/conference requirements, analyzing writing style of target venue, generating venue brief for paper writing, checking submission guidelines. Triggers on: venue research, 期刊调研, 目标期刊, writing style analysis, 写作风格分析, venue requirements, submission guidelines, 投稿要求, journal style, conference format.