Skip to content
Get packs

Guide · Agent skill

How to use the Grade and Iterate Agent Loop agent skill

Updated · written with AI from the project's own documentation and checked against our catalogue data

Grade and Iterate is an agent skill for people building a Claude Managed Agent (CMA). It runs a bounded test-and-improve loop: you write a rubric, a separate grader scores the agent's work, and the skill reads the verdict and recommends the next move. This guide explains what the skill does, how to install it, how to trigger it, and what it needs before you start.

Grade and Iterate Agent Loopalirezarezvani/claude-skills · MIT

Quick start

  1. 1Download and unzip. Download the grade-iterate ZIP from the page and unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder.
  2. 2Check requirements. Confirm Python 3 is installed and that you have a Claude Managed Agent with a session and grader.
  3. 3Write the rubric. Prepare a build-sheet.json with three to five checkable rubric lines, since the rubric is required.
  4. 4Define the outcome. Run outcome_builder.py with --max-iterations set between 1 and 20, then send the payload as a user.define_outcome event.
  5. 5Read the verdict. After each grader run, run verdict_reader.py and act on the recommended move.
  6. 6Run held-back eval. Once a version passes, run eval_scaffold.py to execute held-back cases in parallel, capped at 25 threads.
  7. 7Decide and record. Ship the version or promote it to a scheduled deployment, then record the verdict as the source describes.

What is an agent skill, and what does grade-iterate do?

An agent skill is a folder of instructions and scripts that an AI assistant can load when a task matches it. The skill is described in a SKILL.md file, and the agent uses it when your request fits the description.

grade-iterate is Phase 3 of building a Claude Managed Agent. It covers the grade-then-iterate loop: CMA's outcome primitive grades the agent's work in an isolated context and feeds failing verdicts back for the next attempt. The loop always has a cap set by max_iterations, which must be between 1 and 20. The skill does not try to improve the agent indefinitely.

  • Builds a user.define_outcome payload with a required markdown rubric.
  • Reads each grader verdict and recommends SHIP, SHARPEN, ESCALATE or RESUME.
  • Generates held-back eval cases and a parallel run plan, capped at the 25-thread CMA ceiling.

How do you install the grade-iterate skill?

Download the ZIP for grade-iterate from the page, then unzip it into one of these locations. For a personal setup, use ~/.claude/skills/. For a single project, use the .claude/skills/ folder inside that project.

You can also upload the skill in the Claude apps. Go to Settings → Capabilities → Skills and upload the ZIP there.

  • Personal: unzip into ~/.claude/skills/
  • Project only: unzip into .claude/skills/ in your project folder
  • Claude apps: Settings → Capabilities → Skills, then upload the ZIP

What do you need before you use it?

The skill needs Python 3 and an existing Claude Managed Agent setup with a session and a grader that can receive user.define_outcome events. The skill is only useful inside an agent-building workflow. If you have no agent to build or evaluate, it will not help you.

The skill runs scripts, but according to its source it does not use the network.

  • Python 3
  • A Claude Managed Agent (CMA) with a session and grader
  • A build-sheet.json file for your agent, as used by the scripts

How do you run the three scripts?

The workflow has four stages. First, build the outcome payload from your build sheet with outcome_builder.py. Set max_iterations to a number from 1 to 20. Send the resulting payload as a user.define_outcome event appended to the running session.

Second, after each grader run, read the verdict with verdict_reader.py. It reports the rubric outcome and recommends the next move. Third, each iteration must move at least one rubric line from fail to pass, or the run halts at the cap and escalates. Fourth, once a version passes, run held-back cases with eval_scaffold.py.

  • Define: python3 scripts/outcome_builder.py with --sheet, --max-iterations and --out
  • Read: python3 scripts/verdict_reader.py --result on the last verdict file
  • Evaluate: python3 scripts/eval_scaffold.py with --sheet, --out and --concurrency
python3 scripts/outcome_builder.py \ --sheet ./my-agent/build-sheet.json --max-iterations 5 \ --out ./my-agent/payloads/outcome.json python3 scripts/verdict_reader.py --result ./my-agent/last-verdict.json python3 scripts/eval_scaffold.py \ --sheet ./my-agent/build-sheet.json --out ./my-agent/eval.json --concurrency 5

What do the verdicts mean?

verdict_reader.py maps the grader's result to one of four moves. A satisfied outcome means SHIP. A needs_revision outcome means SHARPEN the agent, then re-run. max_iterations_reached or failed means ESCALATE. interrupted means RESUME.

When only one iteration is left, the skill switches to advice to make the single highest-value fix or escalate now. The grader's explanation should drive the next change, so read it before editing.

  • satisfied → SHIP
  • needs_revision → SHARPEN, then re-run
  • max_iterations_reached or failed → ESCALATE
  • interrupted → RESUME

How do you trigger the skill with natural requests?

The skill loads when you ask for the kind of work it covers. Its description lists these phrases: "grade my agent", "make it pass the rubric", "iterate until it's good" and "is it good enough". It also loads when the orchestrator routes phase=grade-iterate.

Use plain language that names your agent and the rubric. The skill is meant for the grading stage, not for the first launch or for scheduling, which are separate skills.

  • "Grade my agent against the rubric in build-sheet.json."
  • "Make it pass the rubric, but stop after 5 iterations."
  • "Is it good enough to ship? Check the last verdict."
  • "Run the held-back eval cases in parallel."

What safety and limits should you know?

The loop is bounded. Every outcome needs a max_iterations cap, and the cap is clamped to 1 through 20. Held-back cases are never shown to the agent during iteration, so they test generalization.

The grader is a model, and its scores can be wrong. The skill's own guidance says rubric scoring still needs human review. The skill does not use the network, but it does run Python scripts, so read them before you run them on your machine.

  • Always set max_iterations; there is no unbounded mode.
  • Parallel eval runs are capped at 25 threads.
  • Review grader verdicts yourself before shipping.

What people use it for

  • Checking a customer-support agent against a rubric of grounded, checkable answers before you deploy it.
  • Stopping an automated fix loop after 5 attempts so it cannot run up cost.
  • Testing whether an agent that was tuned on examples also handles cases it never saw.
  • Deciding, from a grader's failed verdict, whether to sharpen the prompt or escalate to a person.

Frequently asked questions

Is the grade-iterate skill free?

The skill is published under the MIT licence, as stated in its source. Using it still requires a Claude Managed Agent setup and Python 3, so check the costs of those separately.

What do I need to install grade-iterate?

Download the ZIP, unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder, or upload it under Settings → Capabilities → Skills in the Claude apps. You also need Python 3 and a Claude Managed Agent with a session and grader.

Does grade-iterate need internet access?

According to the source, the skill does not use the network. It does run Python scripts locally, so review them before running.

Can the loop run forever?

No. Every outcome needs max_iterations, clamped between 1 and 20, and the skill stops at that cap and escalates instead of improving indefinitely.

How is grade-iterate different from other agent skills for this workflow?

The source says it is distinct from stage-launch, which handles the first launch, and run-without-you, which handles scheduling. grade-iterate covers only the grading and iteration stage.

Why does the grader sometimes give wrong results?

The grader is a model, and rubric scoring by a model can be wrong. The skill's guidance is that human review is still needed before you trust a verdict.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Grade and Iterate suits developers building Claude Managed Agents who want a capped, structured quality check and held-back tests before deploying. It is not useful if you have no agent to evaluate, and you should not treat the grader as a fully trustworthy judge. If you need open-ended self-improvement, this skill is the wrong tool because it stops at its iteration cap by design.

Download the skill