How to use the Checkpoint Promotion Gate agent skill

What the checkpoint-promotion skill makes the agent do

A trained checkpoint does not ship just because it scored better on its target task. This skill makes the agent walk the checkpoint through four stages in order, and a failure at one stage stops the later stages from running.

Stage 1 is a data-quality check: deduplicating the training set, looking for leaked evaluation answers, and scanning for label noise. Stage 2 re-runs a frozen drift suite (MMLU, GSM8K, IFEval and domain items) and compares the results to a baseline file, using a drift budget. Stage 3 is a paired comparison of the checkpoint against the original base model on the same prompts. Stage 4 is a canary rollout with automatic rollback, which applies only to models that reach production traffic.

The output is a written promotion-report.md ending in a single PROMOTE or REJECT verdict. If the verdict is REJECT, the report names one top fix. The skill does not retrain or repair the model.

Install the checkpoint-promotion skill

Download the skill as a ZIP from the directory page, then unzip it into one of two locations. For a personal setup, use ~/.claude/skills/. For a single project, use the .claude/skills/ folder inside that project.

If you use the Claude apps instead of Claude Code, you can upload the ZIP under Settings → Capabilities → Skills.

• Personal install: unzip into ~/.claude/skills/
• Project install: unzip into the project's .claude/skills/ folder
• Claude apps: Settings → Capabilities → Skills, then upload the ZIP

Check the inputs before you ask for a gate

The skill depends on files that you must already have. Without them, there is nothing to gate against. The source requires a trained model checkpoint, a baseline file eval/baseline-<model>.json produced by the eval-harness-first skill, and a frozen eval/drift-suite.yaml that includes MMLU, GSM8K, IFEval and domain items.

You also need an evaluation harness that can run those benchmarks. The source does not name specific software for this. An LLM judge is needed only if your paired comparison uses a judge; the deterministic variant does not need one.

• Trained model checkpoint
• eval/baseline-<model>.json from eval-harness-first
• Frozen eval/drift-suite.yaml with MMLU, GSM8K, IFEval and domain items
• An evaluation harness able to run the benchmarks
• LLM judge access only if the arena uses judge-based comparison

Trigger the skill with natural requests

You do not need a special command. Describe the situation in plain language and the agent should match the skill from its description. The skill is meant for use after a training run produces a checkpoint, when you are deciding whether a tuned model ships, or when a promoted model must be re-gated against updated goldens.

Name the checkpoint, the baseline file and the drift suite so the agent knows which inputs to use.

• "Gate checkpoint ckpt-0412 against eval/baseline-llama.json before we ship it."
• "Run the promotion gate on this LoRA run and write the promotion report."
• "The goldens changed, so re-gate the promoted model and tell me PROMOTE or REJECT."

Read the drift budget and the verdict

Stage 2 compares each benchmark's drift against the drift budget. A drift of 1 point or less is noise and proceeds. A drift of 2 to 5 points requires a rerun with seed variation, and a drift above 5 points is a hard fail with no exception for task gains. A checkpoint that gained 8 points on its target task but lost 6 points of general capability still fails.

Stage 2 cannot end as RERUN in the final report. A 2 to 5 point drift becomes PROMOTE only if the seed-variation rerun brings it back to 1 point or less; otherwise it becomes REJECT.

Results are reported with a half-width. The source suggests n=200 as a practical floor for item count, which gives roughly ±6 points at typical accuracy. A margin smaller than the half-width is marked REJECT (uncertain), not PASS or HARD FAIL.

• ≤1 pt: noise, proceed
• 2–5 pt: rerun with seed variation before deciding
• More than 5 pt: hard fail

Know what a REJECT means

A REJECT is a correct result from a working gate, not a failed run. Do not rerun the skill just because the verdict was negative.

When rejected, the skill names one remediation from its escalation ladder: first adjust the replay-mix fraction (swap rows rather than adding them), then lower the learning rate, then reduce epochs, then use a smaller LoRA rank. Treat the remediation as a hypothesis for a human to decide on, since the skill does not retrain automatically.