ai-mcp-audit
$
npx mdskill add arcasilesgroup/ai-engineering/ai-mcp-audit``` /ai-mcp-audit scan # coherence analysis (all surfaces) /ai-mcp-audit scan --target <skill-name> # scoped scan (cost-saving) /ai-mcp-audit audit-update <skill-name> # rug-pull detection vs baseline /ai-mcp-audit baseline set --target all # anchor known-good snapshot ```
SKILL.md
.github/skills/ai-mcp-auditView on GitHub ↗
--- name: ai-mcp-audit description: "Audits MCP servers and skills on demand using LLM coherence analysis to catch capability drift and rug-pulls. Trigger for 'audit this skill', 'is this MCP safe', 'check coherence', 'detect rug-pull', 'snapshot baseline', 'mcp audit'. Three modes: scan (declared-vs-observed), audit-update (post-update diff), baseline set (anchor known-good). Not for runtime payload inspection; use prompt-injection-guard hook instead. Not for CVE scanning; use /ai-security instead." effort: high argument-hint: "scan|audit-update [skill]|baseline set [--target skill-or-all]" mode: agent tags: [security, mcp, audit, governance] model_tier: opus mirror_family: copilot-skills generated_by: ai-eng sync canonical_source: .claude/skills/ai-mcp-audit/SKILL.md edit_policy: generated-do-not-edit --- # MCP Audit — On-Demand Skill & MCP Server Security Audit ## Quick start ``` /ai-mcp-audit scan # coherence analysis (all surfaces) /ai-mcp-audit scan --target <skill-name> # scoped scan (cost-saving) /ai-mcp-audit audit-update <skill-name> # rug-pull detection vs baseline /ai-mcp-audit baseline set --target all # anchor known-good snapshot ``` ## Workflow Cold-path LLM-driven security audit (spec-107 D-107-08). Three modes: 1. **Coherence analysis** — declared `description` vs observed code behavior. 2. **Rug-pull detection** — diff post-update files against trusted baseline. 3. **Baseline anchoring** — tamper-evident reference for future audits. Counterpart to **hot-path** runtime control: - **Hot path (Capa 1)** — `prompt-injection-guard.py` PreToolUse hook, $0 cost, deterministic IOC matching, immune to prompt injection of payload (D-107-06). - **Cold path (Capa 2, this skill)** — on-demand LLM analysis. Apt for post-install review, pre-merge audit. Does NOT replace `/ai-security` (CVE/SBOM), `/ai-governance` (compliance), `/ai-verify` (quality). ## When to Use - After installing a new skill or MCP server (`scan`). - After updating an existing skill, especially auto-update (`audit-update <skill>`). - After fresh-cloning or anchoring known-good state (`baseline set`). - Before merging PRs touching `.github/skills/`, `.codex/skills/`, `.agents/skills/`, `.github/skills/`. - NOT for runtime payload inspection (use prompt-injection-guard hook). - NOT for CVE/dependency vulnerabilities (use `/ai-security`). ## Modes ### Mode 1 — `scan` (Coherence Analysis) `ai-mcp-audit scan [--target <path-or-skill-name>]` LLM compares declared `description` vs actual code (handlers, scripts, references). Per surface emits **GREEN (VERDE)** = coherent, or **RED (ROJO)** = suspicious (capability creep, malicious injection, rug-pull). Outputs structured JSON at `.ai-engineering/state/sentinel-scan-report.json` + human-readable stdout. `--target` scopes to single surface (cost-saving). Cost estimate displayed pre-execution; user must confirm. ### Mode 2 — `audit-update <skill>` (Rug-Pull Detection) `ai-mcp-audit audit-update <skill-name>` Reads baseline from `.ai-engineering/state/sentinel-baseline.json` (Mode 3 must run first; without baseline, errors with hint pointing to Mode 3). Walks current files + computes semantic-capability diff: new network calls, new file accesses outside scope, new env reads, new subprocess invocations, new SKILL.md frontmatter capability claims. Each delta flagged with severity (HIGH/MEDIUM/LOW), exact diff, remediation hint. **Postmark-class threat detection pattern** — silent semantic drift that bypasses CVE/SBOM scanning. ### Mode 3 — `baseline set` (Anchoring) `ai-mcp-audit baseline set [--target <skill-name>|all]` Anchors snapshot to `.ai-engineering/state/sentinel-baseline.json`. Per skill: SHA256 of every file + extracted capabilities (network/file/env/subprocess + frontmatter claims). Without baseline, Mode 2 errors. `--target all` regenerates entire baseline (confirmation prompt). Canonical-JSON sort_keys=True for stability + candidate for H2 hash-chained audit trail (D-107-10). ## Triggering Patterns | User intent | Mode | |-------------|------| | "audit this skill", "is this safe?", "check coherence" | `scan` | | post-install of new skill or MCP server | `scan --target <new-skill>` | | "did this rug-pull?", "what changed semantically?" | `audit-update <skill>` | | post-update of existing skill (especially auto-update) | `audit-update <skill>` | | "anchor baseline", "snapshot known-good" | `baseline set` | | post-fresh-clone | `baseline set --target all` | ## Integration `/ai-security` adds CVE/SBOM/secrets; MCP-audit adds coherence/rug-pull. `/ai-governance` consumes VERDE/ROJO verdicts. `/ai-ide-audit` verifies platform support; MCP-audit verifies skill behavior. `prompt-injection-guard` hook (D-107-06) hot-path runtime; MCP-audit cold-path counterpart. ## Non-Goals - No automatic invocation (Q6-3B + OQ-2). Cold-path on-demand only. - No remote MCP server analysis (OQ-4). Local-only in spec-107. - No auto-fix of flagged skills. - No replacement for `/ai-security` (different threat models). ## State Files - `.ai-engineering/state/sentinel-baseline.json` — trusted snapshot (Mode 3 writes; Mode 2 reads). - `.ai-engineering/state/sentinel-scan-report.json` — most recent Mode 1 output. - `.ai-engineering/state/decision-store.json` — risk-acceptance entries for accepted ROJO verdicts (`sentinel-coherence-<skill>` finding-id, spec-105 lifecycle). ## Examples ### Example 1 — coherence scan after installing a new skill User: "I just installed a new skill from a third-party repo, audit it" ``` /ai-mcp-audit scan --target ai-foo-bar ``` Runs LLM coherence analysis comparing the declared `description` against actual handler code. Emits VERDE / ROJO verdict per surface; ROJO triggers risk-acceptance flow. ### Example 2 — rug-pull detection after auto-update User: "did the latest update to ai-skill-x silently change capabilities?" ``` /ai-mcp-audit audit-update ai-skill-x ``` Diffs the current files against the trusted baseline. Reports new network calls, file accesses, env reads, or capability claims with severity HIGH / MEDIUM / LOW. ## References - `.ai-engineering/specs/spec.md` — D-107-08 design rationale. - `.ai-engineering/scripts/hooks/prompt-injection-guard.py` — hot-path runtime counterpart. - `.ai-engineering/security/iocs/IOCS_ATTRIBUTION.md` — IOC catalog provenance.
More from arcasilesgroup/ai-engineering
- ai-adviseProactive governance advisor — checks standards, decisions, and quality trends during development. Always advisory, NEVER blocks. Three modes: `advise` (post-edit), `gate` (pre-dispatch), `drift` (on-demand decision audit). Trigger for 'governance check', 'advise on this change', 'check for drift', 'is this aligned with active decisions', 'shift-left advisory'. Not for blocking gates — use /ai-verify. Not for narrative code review — use /ai-review.
- ai-analyze-permissionsUse when Claude Code keeps asking to approve commands you have already approved, when settings.local.json has grown large, or when you want to consolidate permission grants into wildcard patterns. Trigger for 'too many permission prompts', 'clean up permissions', 'audit my settings', 'consolidate allow rules'. Claude Code only — not available in GitHub Copilot, Antigravity, or Codex.
- ai-animationDesigns motion, transitions, and micro-interactions for UI components: spring animations, gestures, easing, staggers — taste-driven detail compounding. Trigger for 'animate this', 'add transitions', 'micro-interactions for', 'gesture design', 'swipe to dismiss', 'easing for this', 'stagger the'. Not for design systems; use /ai-design instead. Not for visual art; use /ai-visual instead. Not for testing animation code; use /ai-test instead.
- ai-autopilotDelivers large multi-concern specs and backlog runs autonomously: decomposes specs into sub-specs (or normalizes work items into a backlog DAG), deep-plans with parallel agents, builds a dependency DAG, implements in waves, runs a single final quality loop with one bounded quality-remediation pass (verify+guard+review on full changeset), delivers via PR. Trigger for 'implement spec-NNN end to end', 'autopilot this', 'autonomous delivery', 'decompose and ship', 'run the backlog', 'execute these GitHub issues', 'process the sprint backlog'. Invocation is the approval gate. Not for small or single-concern tasks; use /ai-build instead. Not for ambiguous requirements; use /ai-brainstorm first.
- ai-boardOperates the project board (GitHub Projects v2 or Azure DevOps): discovers configuration after install (fields, state mappings, process templates) and syncs work-item state at lifecycle transitions. Trigger for 'set up the board', 'configure our ADO board', 'discover board fields', 'move this issue to in-review', 'update the board', 'mark as in progress', 'sync the work item state'. Two subcommands: `discover` (post-install configuration write) and `sync` (lifecycle state transitions). Auto-invoked via `sync` by /ai-brainstorm, /ai-build, and /ai-pr; fail-open. Not for backlog execution; use /ai-autopilot --backlog instead.
- ai-brainstormForces rigorous design interrogation BEFORE any code: explores approaches, surfaces ambiguity, gathers evidence, produces an approved spec that becomes the contract for /ai-plan. Trigger for 'lets add X', 'how should we handle Y', 'whats the best approach', 'I am thinking about', 'what should we build for'. Not for existing approved specs; use /ai-plan instead. Not for execution; use /ai-build instead.
- ai-branch-cleanupCleans branches safely: switches to the default branch, prunes merged and squash-merged branches, syncs to remote, sweeps stale specs, rotates `.ai-engineering/runtime/` per retention policy. Trigger for 'tidy up', 'tidy branches', 'sync to main', 'delete old branches', 'start fresh', 'rotate runtime'. Auto-invoked by /ai-pr after merge. Not for committing changes; use /ai-commit instead. Not for code-level dead-code removal; use /ai-simplify instead.
- ai-buildCanonical implementation gateway: reads approved plan.md, resolves stack from manifest, deterministic-routes each task to its adapter, dispatches the build agent in an isolated worktree, runs TDD self-validation per task, then a single final quality loop with one bounded quality-remediation pass on the full changeset before /ai-pr. Trigger for 'go', 'start building', 'execute the plan', 'implement it', 'lets do this', 'build the plan', 'resume', 'continue'. Not without an approved plan; run /ai-plan first. Not for multi-concern specs needing decomposition; use /ai-autopilot instead. Not for a single function or subcomponent; use /ai-code.
- ai-codeWrites production code that satisfies stack-context standards on the first pass: interface-first design, backward-compatibility checks, lightweight self-review. Trigger for 'implement this', 'write the code for', 'add X to Y', 'build this function', 'make this work'. Not for tests; use /ai-test instead. Not for debugging; use /ai-debug instead. Not for refactoring; use /ai-simplify instead. Not for executing an approved plan end-to-end; use /ai-build (the gateway).
- ai-commitRuns the governed commit pipeline: auto-branches from protected, stages selectively, formats and lints, scans for secrets, gates docs, composes a conventional message, pushes. Trigger for 'commit my changes', 'save my work', 'push this to remote', 'stage these files', 'ship it'. Not for opening a PR; use /ai-pr instead. Not for branch hygiene; use /ai-branch-cleanup instead.