How to use the ads-test agent skill for paid ad experiments

What the ads-test skill makes the agent do

Before running anything, the agent works through the decision being made, the hypothesis, the control and treatment groups, the main success metric, safety metrics that must not get worse, and when to stop. It also checks platform constraints, overlapping experiments, conversion lag, seasonality, interference, and measurement quality.

When you ask for results, the agent verifies that assignment was handled correctly and that the data is complete before it estimates the effect and its uncertainty. It returns the setup or readout as versioned JSON with a plain-language decision.

• States the decision, hypothesis, treatment, control, randomization unit, population, primary metric, guardrails, minimum effect and stopping rule.
• Calculates sample size and duration from assumptions you declare, and discloses approximations.
• Changes one decision surface at a time unless the design explicitly estimates interactions.
• Warns against peeking and stopping early on a favorable result, calling underpowered noise a winner, or generalizing beyond the tested population.

Install the ads-test skill

Download the ZIP for the skill from the page you are reading. The skill is distributed as a folder containing SKILL.md. Unzip it into one of these locations:

• Personal skills folder: ~/.claude/skills/
• Project skills folder: the .claude/skills/ folder inside your project
• Claude apps: upload the ZIP under Settings → Capabilities → Skills

Trigger the skill with natural requests

You do not need a special command. The skill is meant to be used when your request touches its topics: A/B tests, split tests, experiment design, hypotheses, statistical significance, sample size, test duration, or experiment readouts. Describe your situation in plain language and include the numbers you have, because the agent's estimates depend on the assumptions you give it.

• Help me plan an A/B test on our search ad headlines. Our baseline conversion rate is 3% and we want to detect a 10% relative lift.
• How many days do we need to run a split test on this Meta campaign if we get about 500 clicks per day?
• Here is the data from our experiment: control 120 conversions out of 4,000 clicks, treatment 141 out of 4,050. Is the result trustworthy?
• Review our experiment design before we spend budget. Check the guardrail metrics and stopping rule.

What the skill needs and how it runs

The source lists no required tools or packages. It does not run scripts and does not use the network. The agent works from the information you provide and the instructions in SKILL.md. It does not check real platform data or verify results on its own, so you must supply the figures, and any numbers you paste in are the ones it reasons about.

• Requirements: none listed in the source.
• Runs scripts: no.
• Uses network: no.
• Licence: MIT.

Tips for getting useful output

State your assumptions up front, such as baseline rate, minimum effect worth detecting, traffic volume and the length of your conversion lag. Ask for the setup before launch and the readout after the test, and keep the stopping rule you agreed on. If the agent reports an underpowered result, treat it as inconclusive rather than as a win.

• Write down the decision the test should inform before you start.
• Give the agent complete data, including both groups, not a summary.
• Ask the agent to list the assumptions behind any sample-size estimate.
• Have a qualified person review the statistics before high-stakes spending decisions.