Skip to content
Get packs

Checkpoint Promotion Gate

Decide whether a fine-tuned model ships, using drift budgets, paired comparison and forgetting checks

Source Download skill
Repository
wshobson/agents
Stars
42,095
Licence
MIT
Files
2 files

What it does

This skill makes an AI agent act as a strict release gate for a fine-tuned model. Before a trained checkpoint is shipped, the agent walks it through four stages: checking the training data for leaked test answers and duplicates, re-running a frozen set of general and domain benchmarks to measure capability loss, running a paired comparison against the original base model, and only then recommending a canary rollout. It ends with a written promotion report and a single PROMOTE or REJECT verdict with one top fix if rejected.

Great for

  • ML engineers who fine-tune models with LoRA and need a repeatable go/no-go check before release
  • Teams that already have an evaluation baseline and frozen benchmark suite and want a consistent promotion decision
  • Local-only deployments that need a clear stopping point after offline evaluation

Skip it if

  • Projects without an existing baseline file (eval/baseline-<model>.json) and frozen drift suite, since the skill depends on both as inputs
  • Anyone expecting it to retrain or fix models automatically, because it explicitly does not auto-retrain and only recommends a remediation
  • Quick casual model comparisons, since the gate needs hundreds of benchmark items and multiple runs to produce statistically meaningful verdicts

Try asking

  • “Gate the checkpoint from last night's LoRA run against eval/baseline-llama.json and produce a promotion-report.md”
  • “Check whether this fine-tuned model's drift on MMLU, GSM8K and IFEval is within the 5-point budget before we ship it”
  • “Re-gate our promoted model against the updated goldens and tell me whether it still passes”
  • “Write the Verdict section for this checkpoint and name the single highest-leverage fix if it fails”
Step-by-step guide: how to use Checkpoint Promotion GateInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 2 original files, the MIT licence and an attribution note.

    checkpoint-promotion.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip checkpoint-promotion.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip checkpoint-promotion.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Provide the baseline JSON and frozen drift suite before starting, since the gate cannot run without them
  • Use the deterministic paired-comparison template when every grader is deterministic, and report the half-width of each drift measurement with the verdict
  • If a remediation lever is tried, prefer a seed-variation repeat over trusting a single before/after pair

What's inside

  • references/gate-templates.md
  • SKILL.md

Redistributed unmodified from wshobson/agents under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.