condition-cataloging

$npx mdskill add yogsoth-ai/de-anthropocentric-research-engine/condition-cataloging

Extracts evaluation conditions from a paper to facilitate method comparison and normalization

  • Identifies relevant data splits, hyperparameters, hardware configurations, and seed values in the provided paper.
  • Deploys natural language processing (NLP) techniques to parse through the text for structured information.
  • Utilizes machine learning models trained on similar extraction tasks to accurately identify key conditions without manual intervention.
  • Generates a structured output containing all identified evaluation conditions ready for further analysis or comparison.

SKILL.md

.github/skills/condition-catalogingView on GitHub ↗
---
name: condition-cataloging
description: Record evaluation conditions (data splits, hyperparams, hardware, seeds) from a paper
execution: subagent
prompt: ./prompt.md
input: paper_content, method_name
used-by: baseline-establishment
---

# Condition Cataloging


## Purpose

Extract the complete set of experimental conditions under which a method was evaluated. This metadata is essential for determining whether two scores are directly comparable or require normalization.

## Input Schema

| Field | Type | Description |
|-------|------|-------------|
| paper_content | string | Full paper text (markdown format) |
| method_name | string | The specific method to catalog conditions for |

## Output Schema

```json
{
  "method": "string",
  "conditions": {
    "data": {
      "dataset_version": "string",
      "split": "string",
      "preprocessing": "string",
      "augmentation": "string",
      "training_size": "string",
      "external_data_used": false
    },
    "compute": {
      "hardware": "string",
      "gpu_count": null,
      "training_time": "string",
      "flops_estimate": "string"
    },
    "hyperparameters": {
      "learning_rate": "string",
      "batch_size": null,
      "epochs": null,
      "optimizer": "string",
      "scheduler": "string",
      "key_hyperparams": {}
    },
    "evaluation": {
      "num_seeds": null,
      "seed_values": [],
      "ensemble": false,
      "post_processing": "string",
      "evaluation_protocol": "string"
    },
    "reproducibility": {
      "code_available": false,
      "code_url": "string",
      "pretrained_model_available": false,
      "full_config_provided": false
    }
  },
  "missing_information": ["string"],
  "completeness_score": 0.0
}
```

More from yogsoth-ai/de-anthropocentric-research-engine

SkillDescription
abductive-hypothesis-generationStrategy: 面对异常的最佳解释推理
ablation-brainstormRemove components one by one, observe system changes to reveal hidden dependencies and generate ideas from structural gaps.
ablation-component-mappingMap system architecture to ablatable units for ablation studies
ablation-designDesign ablation studies to isolate component contributions in ML systems
ablation-executionRemove components one by one from a system, record the response/impact of each removal.
abp-vulnerability-classificationClassify assumptions on 2 axes — load-bearing (how much conclusion depends on it) × vulnerable (how likely to be false). Focuses attention on High-Load × High-Vulnerable quadrant.
abstraction-extractionExtract abstract principles from concrete domain cases. Strips domain-specific details to reveal transferable mechanisms.
abstraction-ladderPerform bisociation at multiple abstraction levels
abstraction-ladderingMove between concrete and abstract framings — 3 levels up (Why?) and 3 levels down (How?) to find the most productive research level.
abstraction-to-designAbstract biological principle to design principle. Bridge from biology to engineering.