alterlab-ssci-inference-gate

$npx mdskill add AlterLab-IEU/AlterLab-Academic-Skills/alterlab-ssci-inference-gate

Audits inferential claims for causal language, p-value misuse, and generalization scope.

  • Prevents publishing causal claims without defended identifying assumptions.
  • Consumes Design Passport from alterlab-ssci-design-gate and related skills.
  • Checks p-value and CI interpretations, effect sizes, multiple comparisons, and HARKing.
  • Delivers corrected claims to alterlab-paper-writer or user for revision.

SKILL.md

.github/skills/alterlab-ssci-inference-gateView on GitHub ↗
---
name: alterlab-ssci-inference-gate
description: "Audits final inferential claims against the design, sample, and uncertainty before they are written or published — refuses causal language unless the design's identifying assumption is defended (else downgrades to associational), corrects p-value and confidence-interval misreadings (a p-value is not the probability the null is true, non-significance is not proof of no effect, a 95% CI is not a 95% probability the parameter is inside it), demands effect sizes with intervals rather than significance stars, flags uncorrected multiple comparisons and optional stopping / HARKing, and scopes generalization to the sampling frame. Use when writing or checking a results or discussion section, interpreting a p-value or confidence interval, or deciding whether a finding supports a causal or population claim. For choosing the statistical test prefer alterlab-test-selection-guard; to execute the analysis prefer alterlab-statistical-analysis. Part of the AlterLab Academic Skills suite."
license: MIT
allowed-tools: Read Bash(python:*)
compatibility: No API key required. A discipline-enforcing claim-audit skill; the optional claim linter runs locally via `uv run python` (standard library only).
metadata:
    skill-author: AlterLab
    version: "1.0.0"
    depends_on: "consumes the Design Passport from alterlab-ssci-design-gate / alterlab-ssci-measurement-gate / alterlab-ssci-sampling-gate; hands clean claims to alterlab-paper-writer"
---

# Inference Gate — The Claim May Not Exceed the Design

**Skill type: DISCIPLINE-ENFORCING.** This is the terminal gate of the social-science methods
spine. It reads the accumulated **Design Passport** and audits every inferential sentence against
what actually identifies it — the design, the sample, and the uncertainty. It does not compute; it
refuses conclusions the study cannot support.

## The Core Rule

```
A CLAIM MAY NOT EXCEED ITS DESIGN, ITS SAMPLE, OR ITS UNCERTAINTY.
AUDIT EACH SENTENCE AGAINST WHAT ACTUALLY IDENTIFIES IT.
```

Three ceilings bound every claim. **Design**: causal language is licensed only if the design's
identifying assumption (pinned by `alterlab-ssci-design-gate`) is named and defended — otherwise
the claim is associational. **Sample**: a population generalization is licensed only by a
probability sample of a frame that covers the target population (`alterlab-ssci-sampling-gate`).
**Uncertainty**: a result is reported with an effect size and an interval, interpreted correctly —
not as a bare significant/not-significant verdict. A sentence that breaks any ceiling is rewritten
down to what the study supports.

## When to Use This Skill

- "Help me write / check the results or discussion section."
- "My p-value is 0.03 — what can I conclude?" / "p was 0.20, so there's no effect, right?"
- "Does this finding mean X *causes* Y?"
- "Can I say this holds for [the broader population]?"
- "I ran 30 comparisons and three were significant — what can I report?"

### Does NOT Trigger

| The request is really about… | Route to | Why not this skill |
|---|---|---|
| Which statistical test to run | `alterlab-test-selection-guard` | Test choice, upstream of interpretation. |
| Executing the analysis / computing the estimate | `alterlab-statistical-analysis` | Computation, not claim audit. |
| Choosing the design & identifying assumption | `alterlab-ssci-design-gate` | Design routing happens first. |
| Reliability/validity of the measure | `alterlab-ssci-measurement-gate` | Measurement quality, a different ceiling. |
| Prose polish of an already-sound claim | `alterlab-paper-writer` | Writing, once the claim is audited clean. |

## The three ceilings

### 1. Design ceiling — causal language

Cross-check the claim's verb against `design_type` + `identifying_assumption` in the Passport:

- **Causal verbs** (causes, increases, reduces, improves, leads to, the effect of) require a
  design that identifies a causal effect *and* a defended assumption. Experiment → yes. QED → yes
  **only if** its assumption (parallel trends / exclusion / continuity / no time-varying
  confounders) is defended. Observational without a defended assumption → **downgrade** to
  associational (is associated with, predicts, correlates with).
- Never let a significant coefficient in an observational regression become "X causes Y."

### 2. Sample ceiling — generalization

- **Statistical generalization** to a population requires a probability sample of a covering frame
  (`generalization: statistical` in the Passport). Otherwise scope the claim to the sample
  ("among the surveyed students…") and offer only **analytical** generalization.
- Watch universal quantifiers ("adults", "people generally") attached to a convenience sample.

### 3. Uncertainty ceiling — p-values, CIs, effect sizes

| Misreading | Correction |
|---|---|
| "p = 0.03 means a 3% chance the null is true." | p is P(data this extreme \| null), **not** P(null \| data). |
| "p > 0.05, so there is no effect." | Absence of evidence ≠ evidence of absence. Report the CI; a wide CI around zero is *inconclusive*, not a null proof. |
| "The 95% CI means a 95% probability the true value is in it." | The parameter is fixed; 95% is the long-run coverage of the *procedure*. |
| "It's statistically significant, so it matters." | Significance ≠ importance. Report the **effect size** and its interval; judge practical significance against it. |
| "Three of my 30 tests were significant." | Uncorrected multiplicity. Pre-specify, or correct (FDR / Bonferroni); disclose the family. |
| "I found the hypothesis the data supported." | HARKing / optional stopping. Separate pre-registered confirmatory from exploratory claims. |

A stdlib linter that flags causal verbs, over-broad generalization, and p-value misuse against the
Passport facts: `scripts/claim_audit.py`. Fuller treatment (ASA statement on p-values, New
Statistics, multiplicity): `references/inference_audit.md`.

## Reading the Design Passport

The gate consumes what upstream gates wrote and audits against it:

```yaml
# from design-gate
design_type: observational
identifying_assumption: "conditional ignorability — NOT defended (no plausible ignorability argument)"
claim_type: associational
# from sampling-gate
sampling_method: convenience
generalization: analytical
```

Given this Passport, "remote work **increases** satisfaction **among adults**" fails two ceilings
and is rewritten to "remote work **is associated with** higher satisfaction **among the surveyed
employees**."

## Self-Check Before Publishing

- Does every causal verb trace to a design + a **defended** identifying assumption?
- Is each generalization scoped to what the sample/frame supports (statistical vs analytical)?
- Are results reported as effect size + interval, with p-values interpreted correctly?
- Is a non-significant result framed as inconclusive (CI shown), not as proof of no effect?
- Are multiplicity and any exploratory/optional-stopping caveats disclosed?

## References

- `references/inference_audit.md` — p-value/CI interpretation, multiplicity correction, the three-ceiling audit worked through.
- `scripts/claim_audit.py` — stdlib linter flagging causal-verb, generalization, and p-value overreach against Passport facts.

Part of the AlterLab Academic Skills suite.

More from AlterLab-IEU/AlterLab-Academic-Skills

SkillDescription
alterlab-abm-mesaBuilds agent-based models of social systems with Mesa 3 — the current AgentSet API (model.agents.shuffle_do('step'), auto-assigned unique_id, mandatory super().__init__(seed=...)), cell spaces (mesa.discrete_space OrthogonalMooreGrid / classic mesa.space grids), the DataCollector, batch_run parameter sweeps, and SolaraViz — for emergence, segregation, diffusion, opinion dynamics, and cooperation models. It uses the Mesa 3.x API (the old mesa.time schedulers like RandomActivation are removed) and treats the model as a generative theory to be validated, not just run. Use when the request mentions an agent-based model, Mesa, simulating interacting agents, or emergent macro behavior from micro rules. For discrete-event (queueing/process) simulation prefer alterlab-simpy; for reinforcement learning prefer alterlab-stable-baselines3. Part of the AlterLab Academic Skills suite.
alterlab-academic-careerPrepares academic career documents and professional-development materials for faculty and researchers. Use when drafting academic CVs, research statements, teaching philosophies, diversity statements, faculty-position cover letters, tenure dossiers, promotion narratives, academic portfolios, or mentorship statements, and when planning conference-networking strategy, building academic web presence, setting up ORCID profiles, or understanding impact metrics. Part of the AlterLab Academic Skills suite.
alterlab-accreditation-aolScaffolds program-level Assurance-of-Learning (AoL) documentation for AACSB (2020 Standard 5) and ABET (Criterion 3 Student Outcomes, Criterion 4 Continuous Improvement) accreditation — program learning outcomes / competency goals, curriculum-to-outcome mapping matrices, direct- and indirect-assessment plans, rubric design, and closing-the-loop continuous-improvement narratives — and validates the structure of an outcome-mapping matrix with scripts/aol_matrix.py. Use when the user needs AACSB or ABET assurance-of-learning material, a program-learning-outcomes set, a curriculum/outcome map or coverage matrix, a direct/indirect assessment plan, a closing-the-loop report, or accreditation self-study text. For single-course design or course rubrics prefer alterlab-teaching-design; for post-award grant reports prefer alterlab-grant-reporting. Part of the AlterLab Academic Skills suite.
alterlab-adaptyvSubmits and tracks protein-testing experiments on the Adaptyv Bio Foundry cloud lab (wet-lab validation), and optimizes protein sequences before submission with computational tools (NetSolP, SoluProt, SolubleMPNN, ESM). Use when designing proteins that need wet-lab validation - binding/affinity screening, expression testing, thermostability, or fluorescence assays - or when submitting experiments to the Foundry API, browsing the target catalog, tracking experiment status, retrieving results, or pre-screening sequences for solubility/expression. Triggers on "Adaptyv", "Foundry API", "cloud lab", "biolayer interferometry / BLI", "wet-lab validation". Part of the AlterLab Academic Skills suite.
alterlab-aeonRuns time series machine learning with the aeon library — classification, regression, clustering, forecasting, anomaly detection, segmentation, and similarity search via scikit-learn compatible APIs. Use when working with temporal data, sequential patterns, or time-indexed observations (univariate or multivariate) that need specialized algorithms beyond standard ML approaches. Part of the AlterLab Academic Skills suite.
alterlab-akademik-tesvikComputes the annual Turkish academic-incentive (akademik teşvik) score for Devlet (state) university öğretim elemanı (academic staff) from a classified activity list, applying the verified Akademik Teşvik Ödeneği Yönetmeliği (2018/11834) Faaliyet Hesaplama Tablosu headline puanları (Proje 20, Araştırma 15, Yayın 30, Patent 30, Atıf 30, Tebliğ 20, Ödül 20) with the k (author-count), p (Q1 1/Q2 0.8/Q3 0.5/Q4 0.25), and r (project-role) coefficients via türü puanı = Σ(faaliyet oranları) × headline (MADDE 8/2), then enforcing the per-type headline ceilings, the 100 cap (MADDE 8/3), and the net-30 payment gate (MADDE 10/3) in a deterministic scripts/tesvik_score.py. Use when the user wants to calculate akademik teşvik puanı, check the en az 30 (net-30) threshold, apply TABLO 4 ceilings, or estimate teşvik eligibility; keeps its coefficients separate from doçentlik scoring. For doçentlik point eligibility prefer alterlab-docentlik-eligibility. Part of the AlterLab Academic Skills suite.
alterlab-alpha-vantageAccesses real-time and historical stock market data, forex rates, cryptocurrency prices, commodities, economic indicators, and 50+ technical indicators via the Alpha Vantage API (requires a free API key from alphavantage.co). Use when fetching stock prices (OHLCV), company fundamentals (income statement, balance sheet, cash flow), earnings, options data, market news/sentiment, insider transactions, GDP, CPI, treasury yields, gold/silver/oil prices, Bitcoin/crypto prices, forex exchange rates, or calculating technical indicators (SMA, EMA, MACD, RSI, Bollinger Bands). Part of the AlterLab Academic Skills suite.
alterlab-alphafoldPredict protein 3D structures with AlphaFold2 via ColabFold — MMseqs2-accelerated MSAs, monomer and AlphaFold2-Multimer complex folding, and confidence-based validation (pLDDT, pTM/ipTM, PAE). Use when folding a protein sequence or complex from FASTA, generating a predicted structure with confidence metrics, ranking models, or checking self-consistency of a design. For co-folding a protein WITH a small-molecule ligand or predicting binding affinity prefer alterlab-boltz; for antibody–antigen or one-FASTA multi-entity complexes prefer alterlab-chai; to LOOK UP an already-computed structure prefer alterlab-alphafold-db; for ESM embeddings or inverse folding prefer alterlab-esm. Part of the AlterLab Academic Skills suite.
alterlab-alphafold-dbAccess the AlphaFold DB of 200M+ AI-PREDICTED protein structures — retrieve models by UniProt accession, download PDB/mmCIF files, and analyze prediction confidence metrics (pLDDT, PAE). Use when a UniProt ID needs a computationally predicted 3D structure or when no experimental structure exists, for homology modeling, protein engineering, or structure-based drug discovery; for EXPERIMENTALLY determined structures (X-ray, cryo-EM, NMR) prefer alterlab-pdb, and for protein sequences, annotations, or accession ID mapping prefer alterlab-uniprot instead. Part of the AlterLab Academic Skills suite.
alterlab-anndataBuild, slice, concatenate, read, and write AnnData annotated data matrices (obs, var, X, layers, obsm, uns) — the scverse data STRUCTURE, not an analysis pipeline. Use when creating or wrangling .h5ad/zarr files, managing cell and gene annotations, concatenating batches, or handling layers/obsm/backed-mode; for the QC, normalization, clustering, UMAP, and differential-expression analysis pipeline prefer alterlab-scanpy instead, and for RNA velocity from spliced/unspliced layers prefer alterlab-scvelo instead. Part of the AlterLab Academic Skills suite.