absolutelyskilled/absolute-deflake
skills/absolute-deflake/SKILL.md
Flaky test fixes: detect nondeterministic tests empirically (repeat/shuffle/parallel runs), diagnose the root cause, fix it — never retry/skip/sleep — and verify across many randomized runs. Triggers on "absolute deflake", "fix flaky tests", "CI is flaky", "this test fails randomly/intermittently".
$ install --global
skillsauthnpx skillsauth add absolutelyskilled/absolutelyskilled absolute-deflakeInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Jun 29, 2026, 4:20 AM187.9s3 files scanned
SKILL.md
- name:
- absolute-deflake
- version:
- 0.5.0
- description:
- >
- Flaky test fixes:
- detect nondeterministic tests empirically (repeat/shuffle/parallel runs), diagnose the root cause, fix it — never retry/skip/sleep — and verify across many randomized runs.
- category:
- workflow
- user-invocable:
- true
- argument-hint:
- [target]
- license:
- MIT
- - github:
- maddhruv
Start your first response with the 🧪 emoji.
Absolute Deflake
Find tests that pass and fail nondeterministically, diagnose the root cause of each, and fix it — not by retrying or skipping, but by removing the source of nondeterminism. Output is evidence (failure rate per test) → cause → fix, verified by repeated runs.
Runs the shared engine in references/health-engine.md — read it for the
DETECT → SCAN → TRIAGE → FIX → VERIFY → REPORT loop and the safety contract. This file
covers only what's specific to flaky tests.
When to use
- "Our CI is flaky", "this test fails randomly", "fix the intermittent failures".
- A test passes locally but fails in CI (or vice versa), or fails ~1 in N runs.
- Burning down a backlog of
retry/skip-marked tests that mask real flakiness.
Not for tests that fail deterministically — that's a real bug or a real regression
(/absolute work for a fix, or just fix it). deflake targets nondeterministic failures.
What it scans
Establish flakiness empirically — a test isn't flaky because someone said so. Use
preferences.health.deflakeRuns from config as the default N for repeat-runs (else 20):
| Ecosystem | Repeat-run / detect |
|---|---|
| Jest/Vitest | run suite N× (--run loop), randomize order (--shuffle / testSequencer) |
| pytest | pytest-randomly + pytest --count=N (pytest-repeat); -p no:randomly to A/B |
| Go | go test -count=N -shuffle=on ./..., -race |
Also mine signals: existing retry/flaky/skip annotations, CI history if reachable, and
run the suite both in isolation and in full/parallel — order- and concurrency-
dependent failures only show one way. Record a failure rate per suspect test.
Common root causes (diagnose, don't guess)
| Cause | Tell | Fix |
|---|---|---|
| Test-order / shared state | passes alone, fails in suite (or vice versa) | isolate state; reset/teardown between tests |
| Time / clock | fails near midnight, DST, or under load | fake timers / inject clock; no real sleep |
| Async race / missing await | fails under parallelism or slow CI | await the actual condition; no fixed timeouts |
| Randomness | fails ~X% with no pattern | seed the RNG; fix the seed in tests |
| Network / external I/O | fails offline or on slow links | mock/stub the boundary |
| Unordered collections | fails on map/set iteration order | sort before asserting |
| Resource leak / port reuse | fails on repeat or parallel runs | unique resources; clean up |
Risk ranking (TRIAGE)
| Wave | Class | Default | |---|---|---| | 1 | clear, isolated cause (seed, await, fake clock, sort) | fix now | | 2 | shared-state / ordering — needs fixture refactor | fix this pass, per test | | 3 | flakiness pointing at a real product race, not just the test | gated — surface; may be a genuine bug to fix in code |
A flaky test sometimes means the code has a race, not the test. Don't "stabilize" the test into hiding a real concurrency bug — flag wave-3 cases for a real fix.
Fix & verify
- Fix the cause. Then prove it: re-run the test many times (and shuffled / parallel /
with
-race) — green once is not deflaked; green across N randomized runs is. - Remove the
retry/skip/flakyannotation that was masking it once the cause is fixed. - Never "fix" by adding retries, raising timeouts blindly,
sleep, or skipping the test — that hides flakiness, doesn't remove it. - Re-run the full suite to confirm the fix didn't destabilize neighbors.
Gotchas
- Retry/skip as a fix. Masks the flake, ships the nondeterminism. Forbidden here.
sleepto dodge a race. Slows the suite and still flakes under load. Await the condition.- One green run = done. Flakes are probabilistic — verify with many randomized runs.
- Stabilizing a real product race. If the code races, fix the code, not just the assertion.
- Ignoring order/parallel dimension. Run isolated and in-suite; the bug hides in whichever you skip.
Companion commands
/absolute upgrade— a flaky suite makes upgrade verification unreliable; deflake first./absolute debt— flaky-test annotations are test debt; this clears them at the root./absolute work— when the flake is a genuine product-code race needing real design.
Related Skills
absolutelyskilled/absolute-simplify
development
VerifiedTrustedCommunity
Use when the user wants to simplify, clean up, refactor, tidy, or refine code — their staged/unstaged git changes or a target file/path. Reduces complexity, flattens nesting, removes redundancy and dead code, scores each change by value (holding low-value churn), then runs tests to prove nothing broke. Invoke on: "simplify", "simplify this", "simplify my code/changes", "clean up", "clean this up", "clean up my changes", "refactor this", "make this cleaner", "tidy this up", "reduce complexity", "flatten this", "remove dead code", "make it more readable", "polish before commit", or "absolute simplify". Acts on your working diff; for repo-wide dead code use absolute-prune; for lint/type debt use absolute-debt.
192SKILL.mdUpdated Apr 11, 2026
absolutelyskilled/absolute-upgrade
testing
VerifiedTrustedCommunity
Dependency upgrades: outdated/vulnerable deps planned into semver waves (patch/minor batched, majors gated and changelog-read), applied incrementally with lockfiles regenerated and tests green after each. Runs on green main. Triggers on "absolute upgrade", "upgrade our dependencies", "bump deps", "update packages", "move off the deprecated X", "clear the Dependabot backlog".
190SKILL.mdUpdated Jun 25, 2026
absolutelyskilled/absolute-spec
development
VerifiedTrustedCommunity
Lightweight standalone design spec for AI coding agents: codebase scan → bounded clarify pass (3–5 questions, not a grill) → reviewed design doc written to docs/plans/ → independent scored review → stop. No task board, no build. Use when you want a spec to discuss, hand off, or review before committing to implementation. Chains into absolute-work when ready to build. Triggers on "absolute spec", "write a spec", "spec out this feature", "draft a design doc", "I want a spec to hand off / review, don't build it yet".
190SKILL.mdUpdated Jun 25, 2026
absolutelyskilled/absolute-prune
tools
VerifiedTrustedCommunity
Dead code and dependency cleanup, repo-wide: unused deps, unreferenced exports, unreachable code, orphaned files — removed only with tool evidence, in reversible waves. Runs on green main. For diff-scoped cleanup use absolute-simplify. Triggers on "absolute prune", "remove dead code", "find unused deps/exports", "what can we delete", "clean up orphaned files".
190SKILL.mdUpdated Jun 25, 2026