file-test-bug

$npx mdskill add microsoft/GitHub-Copilot-for-Azure/file-test-bug

Diagnose test failures and create GitHub issues.

  • Automatically generates bug reports for integration test failures.
  • Reads JUnit XML and agent metadata to analyze failures.
  • Determines root causes by comparing agent behavior to test assertions.
  • Outputs detailed explanations and fix suggestions before creating issues.

SKILL.md

.github/skills/file-test-bugView on GitHub ↗
---
name: file-test-bug
description: "File a GitHub issue for local integration test failures. TRIGGERS: file test bug, report test failure, create bug for test, integration test failed, test failure issue, junit failure"
license: MIT
metadata:
  author: Microsoft
  version: "1.0.0"
---

# File Test Bug

Creates a GitHub issue in `microsoft/github-copilot-for-azure` for integration test failures.

## Input

- **Skill name** (required): e.g., `azure-rbac`, `appinsights-instrumentation`
- **Test run** (optional): Timestamp of test run. Defaults to most recent with logs for the skill.

## Steps

1. Ask user for skill name if not provided
2. Parse `tests/reports/junit.xml` for failures matching the skill
3. Find test run directory (specified or most recent with matching logs)
4. Read `agent-metadata.md` from `tests/reports/test-run-<timestamp>/<skillname>-<testname>/`
5. For each failure, read the actual line of code from the test file using the location (file:line) from junit.xml
6. **REQUIRED - Write diagnosis BEFORE creating issue:**
   - Analyze the agent-metadata.md to understand what the agent did
   - Compare agent behavior to what the test expected (from the assertion)
   - Identify the root cause (skill issue, test issue, or model behavior)
   - Write 2-3 sentences per failed test explaining WHY it failed
   - Suggest potential fixes (update skill, update test, or update fixtures)
7. Create issue via `github-mcp-server-create_issue`:

```
owner: microsoft
repo: github-copilot-for-azure
title: Integration test failure in <skill-name>
labels: ["bug", "integration-test"]
body: |
  ## Failed Tests
  - <test-name>: <error message>
  
  ## Diagnosis
  
  ### Root Cause
  <1-2 sentences explaining WHY the test failed based on agent-metadata.md analysis>
  
  ### Analysis per Test
  - **<test-name>**: <what agent did vs what test expected>
  
  ### Suggested Fix
  <one of: update skill, update test assertions, provide test fixtures>

  ## Details
  ### <test-name>
  **Error:** <failure from junit.xml>
  **Location:** <file:line>
  ```typescript
  <actual line of code from the test file at the specified line number>
  ```
  
  <details>
  <summary>agent-metadata.md</summary>
  
  <full contents of agent-metadata.md file, verbatim>
  
  </details>
```

## Important

Include the **complete, unmodified contents** of each `agent-metadata.md` file in the issue body. Do NOT summarize or truncate the logs. Wrap each log in a `<details>` block with the test name as the summary.

More from microsoft/GitHub-Copilot-for-Azure

SkillDescription
analyze-skill-issuesQuery the integration-test storage account to find why a specific skill's tests are failing. Reads blob-stored test result files and surfaces error details. TRIGGERS: why is skill failing, skill test failures, debug skill tests, skill failing tests, analyze skill failures, why are tests failing for skill, skill test errors, investigate skill issues. DO NOT USE FOR: analyzing a GitHub Actions run report or comparing test runs across runs (use analyze-test-run).
analyze-test-runAnalyze a GitHub Actions integration test run and produce a skill invocation report with failure root-cause issues. TRIGGERS: analyze test run, skill invocation rate, test run report, compare test runs, skill invocation summary, test failure analysis, run report, test results, action run report
azure-hosted-copilot-sdkBuild, deploy, modify GitHub Copilot SDK apps on Azure. MANDATORY when codebase contains @github/copilot-sdk or CopilotClient — use this skill instead of azure-prepare. PREFER OVER azure-prepare when codebase contains copilot-sdk markers. WHEN: copilot SDK, @github/copilot-sdk, copilot-powered app, deploy copilot app, add feature, modify copilot app, BYOM, bring your own model, CopilotClient, createSession, sendAndWait, azd init copilot. DO NOT USE FOR: general web apps without copilot SDK (use azure-prepare), Copilot Extensions, Foundry agents (use microsoft-foundry).
azure-rbacHelps users find the right Azure RBAC role for an identity with least privilege access, then generate CLI commands and Bicep code to assign it. Also provides guidance on permissions required to grant roles. WHEN: bicep for role assignment, what role should I assign, least privilege role, RBAC role for, role to read blobs, role for managed identity, custom role definition, assign role to identity, what role do I need to grant access, permissions to assign roles.
investigate-integration-testInvestigate a failing integration test from a GitHub issue. Downloads logs/artifacts, analyzes the failure, examines relevant skills, and suggests fixes. TRIGGERS: investigate integration test, debug integration test, failing integration test, test failure investigation, diagnose test failure, analyze test issue
markdown-token-optimizerAnalyzes markdown files for token efficiency. TRIGGERS: optimize markdown, reduce tokens, token count, token bloat, too many tokens, make concise, shrink file, file too large, optimize for AI, token efficiency, verbose markdown, reduce file size
sensei**WORKFLOW SKILL** — Iteratively improve skill frontmatter compliance using the Ralph loop pattern. WHEN: \"run sensei\", \"sensei help\", \"improve skill\", \"fix frontmatter\", \"skill compliance\", \"frontmatter audit\", \"score skill\", \"check skill tokens\". INVOKES: token counting tools, test runners, git commands. FOR SINGLE OPERATIONS: use token CLI directly for counts/checks.
skill-authoringGuidelines for writing Agent Skills that comply with the agentskills.io specification. WHEN: \"create a skill\", \"new skill\", \"write a skill\", \"skill template\", \"skill structure\", \"review skill\", \"skill PR\", \"skill compliance\", \"SKILL.md format\", \"skill frontmatter\", \"skill best practices\".
skill-reviewerReview skill PRs with structured severity-rated feedback covering token budgets, routing conflicts, required sections, and repo conventions. WHEN: \"review skill\", \"review skill PR\", \"review skill changes\", \"check skill quality\", \"skill PR feedback\".
submit-skill-fix-prSubmit a pull request with skill fixes. Validates skill structure, bumps versions, and creates a PR with a proper description. TRIGGERS: submit skill fix, create fix PR, skill fix pull request, submit PR, push skill fix