How to use the experimental-design agent skill: setup and examples

What does the experimental-design skill make the agent do?

The skill contains only text guidance in a SKILL.md file. It does not run experiments or check results. When the agent is asked to design a test or comparison, the guidance pushes it to follow a fixed set of practices.

• Include meaningful baselines: at least one classical method, at least one recent state-of-the-art method, and a simple but strong baseline such as a linear probe or k-NN.
• Run experiments with multiple random seeds, minimum 3 and ideally 5.
• Report the mean and standard deviation across seeds.
• Design ablations that remove one component at a time, each meaningfully different from the baseline.
• Change only one thing per comparison.
• Use standard train/validation/test splits and never test on training data.
• Report wall-clock time and memory usage alongside accuracy.

How do you install the experimental-design skill?

You can install the skill in either of two ways. Download the ZIP file from this page first. The skill's only listed file is SKILL.md, so the ZIP should contain that file in a folder named after the skill.

• Option A, personal install: unzip the download into ~/.claude/skills/ so the skill is available across your projects.
• Option B, project install: unzip it into the .claude/skills/ folder inside a specific project so it applies only there.
• Option C, Claude apps: open Settings, go to Capabilities, then Skills, and upload the ZIP.

How do you trigger the skill with natural requests?

You do not need a special command. The skill is described as applying when you are planning ablations, baselines or controlled experiments, and its metadata lists trigger keywords such as experiment, ablation, baseline, control, hypothesis and reproducibility. Ask for the design in ordinary language and mention one of these ideas so the agent knows which practices to apply.

• Plan an ablation study for my new text classifier and include the baselines I should compare against.
• Help me design a controlled experiment to test whether the new data augmentation helps.
• Outline the baselines and seeds needed to make this benchmark reproducible.

What does a good result look like?

After you ask for a design, check that the agent's plan covers the core practices. A useful plan names specific baselines, states how many seeds it will use, says which component each ablation removes, lists the single variable that changes in each comparison, describes the train, validation and test split, and includes time and memory measurements in the reported results. If one of these is missing, ask the agent to add it. The skill guides the plan; you still need to run the experiments and verify the numbers yourself.

What does the skill need, and what are its safety notes?

The skill lists no requirements. It does not run scripts and does not use the network, so it only shapes the agent's written reasoning and plans. Because it contains no executable code, there are no script permissions to review. It is licensed under MIT.

• Requires: nothing listed.
• Runs scripts: no.
• Uses network: no.

Tips and when not to use it

The guidance is specific to machine learning benchmarks, so rules for non-ML experiments or lab work would not transfer cleanly. For quick exploratory coding, multi-seed evaluation is usually unnecessary overhead. The skill is also one of the stages used by the AutoResearchClaw research pipeline, where it is applied at experiment-design stages.

• Use it for planning fair comparisons, reproducible benchmarks and ablation designs in ML.
• Skip it for quick exploratory code or non-ML lab work.
• Treat the skill as a checklist, not a verifier of the agent's work.