cwilliams5/review-test-skips
.claude/skills/review-test-skips/SKILL.md
Audit all SKIP statements in the test suite for correctness
$ install --global
skillsauthnpx skillsauth add cwilliams5/Alt-Tabby review-test-skipsInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Apr 17, 2026, 1:17 AM14.1s1 file scanned
SKILL.md
- name:
- review-test-skips
- description:
- Audit all SKIP statements in the test suite for correctness
- user-invocable:
- true
- disable-model-invocation:
- true
Audit every Log("SKIP: statement across all .ahk test files in tests/. For each one, determine whether it is a legitimate skip or a silent failure that should be FAIL + TestErrors++.
Classification Rules
A SKIP is legitimate when the test is genuinely not applicable in the current environment:
- Optional dependency not installed: komorebi not installed/running, komorebic.exe not found
- Compilation not done: compiled exe not found (smart-skip scenario)
- Elevation required: test needs admin and process isn't elevated
- Environmental data insufficient: not enough windows open, no multiwindow class found, system process not running
- Parallel worktree collision: mutex conflict from another test instance running concurrently (worktree-safe design)
A SKIP is a silent failure (should be FAIL) when test infrastructure that SHOULD be working isn't:
- Process startup failed (launcher/gui/pump didn't spawn)
- Pump not connected when setup confirmed it connected
- GUI process not found after successful launch
- Any precondition that the test's own setup was supposed to guarantee
Key Principle
If the test set up the condition itself (launched the exe, connected the pump, etc.) and then can't find it — that's a FAIL, not a SKIP. If the condition is external to the test (komorebi installed, admin rights, enough windows on desktop) — that's a legitimate SKIP.
Steps
- Search all
tests/*.ahkfiles forLog("SKIP:andLog('SKIP:patterns - For each occurrence, read the surrounding context (condition, whether
TestErrors++follows, whether it returns early) - Classify each as legitimate or silent failure using the rules above
- Report findings in a table grouped by file:
| File | Line | Condition | Verdict | Issue | |------|------|-----------|---------|-------| | ... | ... | ... | OK / FAIL | description if FAIL |
- If any silent failures are found, fix them: change
SKIPtoFAILand addTestErrors++ - Run
.\tests\test.ps1 --liveto confirm all tests still pass after changes
Ignore any existing plans — create a fresh one.
Related Skills
cwilliams5/worktree
tools
VerifiedTrustedCommunity
Create a new git worktree and switch the session into it
1SKILL.mdUpdated Apr 16, 2026
cwilliams5/trace
tools
VerifiedTrustedCommunity
Spawn agent to trace code flow via query tools — answer only, no context cost
1SKILL.mdUpdated Apr 16, 2026
cwilliams5/ship
tools
VerifiedTrustedCommunity
Commit, push, and create a PR for the current branch
1SKILL.mdUpdated Apr 16, 2026
cwilliams5/shader-retire
tools
VerifiedTrustedCommunity
Retire a shader by moving its files to legacy/shaders_retired
1SKILL.mdUpdated Apr 16, 2026