Guide · Agent skill
How to use the GRPO & RLVR Training agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The GRPO & RLVR Training skill is a set of instructions that an AI agent loads so it can set up reinforcement-learning fine-tuning for language models on tasks where success can be checked automatically, such as math answers, code that passes tests, or valid structured output. It is written for ML engineers and developers who already have a verifier for their task. After this guide you will know how to install the skill, how to ask for it in plain language, what you need before starting, and which safety checks the skill expects you to run.
GRPO & RLVR Trainingwshobson/agents · MIT Download the skill Quick start
- 1Download and unzip the skill. Download the ZIP from this page and unzip it into ~/.claude/skills/ for a personal install or into .claude/skills/ inside your project.
- 2Confirm the skill loads. Check that the skill appears under Settings, then Capabilities, then Skills in the Claude apps, or that the project folder is in place for Claude Code.
- 3Check that RL fits your task. Confirm the task has an automatic pass/fail check and that the base model already succeeds sometimes. If it never succeeds, do SFT first.
- 4Write and inspect the reward function. Ask the agent to build a composite reward, then run it on 50 to 100 sampled outputs and read the scores yourself before training.
- 5Run the GRPO config. Start training with the validated GRPOConfig and watch for entropy collapse, length-reward correlation or MoE instability before choosing a variant.
What the skill makes the agent do
Once installed, the agent follows a structured recipe for training with GRPO (group relative policy optimization) and reinforcement learning from verifiable rewards (RLVR). It helps decide whether RL is the right tool at all, since GRPO only pays off when a unit test passes, a parser accepts the output, or an answer matches a ground truth.
The skill covers a GRPO training configuration using TRL's GRPOTrainer with vLLM generation, a required manual check of reward-function scores before training starts, and guidance on when to switch to variants such as DAPO, Dr.GRPO or GSPO. It also points to reference files containing reward-function code and memory sizing by model size class.
- Decides whether the task is verifiable enough for RL
- Produces a GRPO config and reward-function structure based on the skill's references
- Requires reward inspection on 50 to 100 sampled outputs before training
- Suggests a variant only when a specific failure mode appears
Install the skill from the ZIP or the Claude apps
Download the ZIP for this skill from this page. Then choose one of the two install locations below. A personal install makes the skill available across your projects, while a project install keeps it limited to one repository.
For a personal install, unzip the file into ~/.claude/skills/. For a single project, unzip it into the .claude/skills/ folder inside that project. You can also upload the skill in the Claude apps under Settings, then Capabilities, then Skills.
- Personal: unzip into ~/.claude/skills/
- Project: unzip into .claude/skills/ inside your project
- Claude apps: Settings → Capabilities → Skills, then upload
Trigger the skill with natural requests
You do not need a special command. The skill is meant to load when your request matches its description, which covers tasks where success is algorithmically checkable, designing GRPO reward functions, and runs that diverge or reward-hack. Describe the task and the check you have in mind.
Requests that fit well name the verifier, for example a test suite, a schema, or an answer checker. Vague requests about subjective quality do not match, because the skill directs those to evaluation and preference methods instead.
- 'I have a math dataset with ground-truth answers. Should I use GRPO to improve my SFT checkpoint?'
- 'Design reward functions for a model that must return valid JSON tool calls.'
- 'My GRPO run reward keeps rising but outputs got worse. Help me find the reward hack.'
- 'Write a GRPO config for a single 24GB GPU for a small model.'
Check the requirements before you start
The skill assumes several things are already in place. You need Python with TRL (GRPOConfig and GRPOTrainer), vLLM for generation, and GPU hardware. Memory sizing details for each size class are in references/grpo-memory.md.
You also need a verifier for your target task, such as a code executor, test suite, schema checker or grader, and an SFT checkpoint to use as the starting model. The skill says to do supervised fine-tuning first if the model never succeeds on the task.
- Python with TRL (GRPOConfig, GRPOTrainer)
- vLLM for generation (use_vllm=True)
- GPU hardware
- A verifier for the target task
- An SFT checkpoint as the starting model
Follow the recipe and the reward-inspection gate
The reference recipe passes use_vllm=True and sets vllm_mode to colocate on a single GPU or server for multi-GPU setups. It uses num_generations of at least 8, learning_rate=5e-7 and beta=0.01 as the settled starting values, and a composite reward that combines a format reward with a correctness reward.
Before the real run, the skill requires you to run the reward function against 50 to 100 sampled outputs and read the results yourself. If the reward disagrees with your reading, fix the reward first. Reward-function implementations for exact match, schema validation, unit-test execution, a length-penalty wrapper and a rubric-style judge are in references/reward-functions.md.
- Keep num_generations at 8 or higher
- Use a composite reward: format plus correctness
- Inspect 50 to 100 outputs before training
- Change learning_rate or beta only after a stable, inspected base run
grpo_args = GRPOConfig(
output_dir="./outputs-grpo",
use_vllm=True,
vllm_mode="colocate",
num_generations=8,
learning_rate=5e-7,
beta=0.01,
bf16=True,
seed=3407,
)Safety notes on scripts and network
This skill runs no scripts and uses no network access, according to its listing. It consists of instructions and reference files that the agent reads. The training code and reward functions you run are your own responsibility, so review them before executing, especially reward functions that run generated code such as unit-test execution.
Because a reward function that runs model-generated code can be risky, keep execution in a controlled environment. The skill itself does not grant that environment to you.
- Runs scripts: no
- Uses network: no
- Review all reward-function and training code before running it
What people use it for
- Improving math answer accuracy by rewarding correct final answers that match a ground truth.
- Training a model to produce code that passes an existing unit test suite.
- Teaching a model to output valid JSON tool calls checked by a schema validator.
- Diagnosing a GRPO run whose reward rises while output quality falls.
Frequently asked questions
How do I install the GRPO & RLVR Training skill?
Download the ZIP from this page and unzip it into ~/.claude/skills/ for a personal install, or into .claude/skills/ inside a project. You can also upload it in the Claude apps under Settings, then Capabilities, then Skills.
Is the grpo-rlvr-training skill free to use?
The skill is published under the MIT licence, according to its listing. The licence covers the skill files, but you still need your own Python, TRL, vLLM and GPU setup to run training.
What do I need to use this skill?
You need Python with TRL, vLLM for generation, GPU hardware, a verifier for your task such as a test suite or schema checker, and an SFT checkpoint as the starting model. Memory details for each model size class are in references/grpo-memory.md.
When should I use GRPO instead of supervised fine-tuning?
Use GRPO when the model already succeeds sometimes on a task that can be checked automatically. If the model never succeeds, the skill says to do supervised fine-tuning first, and use preference methods when the target is a preference between acceptable outputs.
What should I do if my GRPO run reward-hacks or diverges?
First re-read your reward function against 50 to 100 sampled outputs, since an uninspected reward is a common cause. The skill also describes variants: DAPO for entropy collapse on long reasoning, Dr.GRPO when length rises regardless of quality, and GSPO for mixture-of-experts models.
Does this skill work for vision-language models?
No. The skill states that vision-language RL is not executed as a runnable path in v1. It is documented only for context and is treated as a research spike outside the supported recipe.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
This skill suits ML engineers who have a reliable automatic check for their task and a model that already succeeds sometimes. If your target depends on subjective judgment or the model never succeeds, pick an evaluation or supervised fine-tuning approach instead.
Download the skill