Deflake a Flaky Test
Makes a flaky test reliably pass with a minimal fix that keeps its checks intact
- Repository
- QwenLM/qwen-code
- Stars
- 28,880
- Licence
- Apache-2.0
- Files
- 1 file
What it does
When a test passes on one run and fails on another with the same code, it is called flaky. This skill tells an AI coding agent how to find the cause of that randomness in a specific test and apply one of four small, allowed fixes, such as raising a timeout, waiting for a real condition instead of a fixed delay, seeding randomness or fake timers, or giving colliding tests separate resources. It explicitly forbids deleting, skipping, or weakening the test's assertions. If the failure looks like a real product bug, the agent is told to write a failure report and stop for a human.
Great for
- Maintainers of the qwen-code repository triaging recurring CI failures on a named test
- Developers who want an agent to make one flaky Vitest test deterministic without changing what it checks
Skip it if
- Tests that fail consistently: the skill targets intermittent failures, and a stable failure points to a real bug (the skill itself says to escalate those)
- Projects that do not use Vitest or the qwen-code prepare-pr workflow, since the instructions assume those tools and paths exist
Try asking
- “Fix the flaky test in the deflake issue that says Test timed out in 5000ms in the cli package”
- “Stabilize the order-dependent test in the tempdir helper tests and run the file several times to prove it passes”
- “Deflake the test that depends on Math.random and pin the seed”
Install in 3 steps
- 1
Download the skill
The ZIP contains the 1 original files, the Apache-2.0 licence and an attribution note.
deflake.zip - 2
Add it to your agent
Claude Code (all projects)
mkdir -p ~/.claude/skills && unzip deflake.zip -d ~/.claude/skills/Claude Code / Agent SDK (this project only)
mkdir -p .claude/skills && unzip deflake.zip -d .claude/skills/Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.
- 3
Ask for the task
No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.
Get better results
- Give the agent a specific deflake issue that names one test file and test name, since the skill is built around a single test
- Ask for the repeated-run evidence, because the skill requires the test to pass every time across several runs before it counts as fixed
- Review the diff afterwards to confirm no assertion, input, or expected value changed
What's inside
- SKILL.md
Redistributed unmodified from QwenLM/qwen-code under the Apache-2.0 licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.