Skip to content
Get packs

Guide · Agent skill

How to use the slime RL Training agent skill

Updated · written with AI from the project's own documentation and checked against our catalogue data

The slime RL Training agent skill is a guide that gives an AI agent step-by-step knowledge of slime, a framework for reinforcement-learning post-training of large language models. It pairs Megatron-LM for training with SGLang for rollout generation. This page explains what the skill does, how to install it, how to ask for it in plain language, and what you need before running anything. After reading, you should be able to load the skill and ask your agent to walk you through a GRPO run or an asynchronous training setup.

slime RL Training GuideOrchestra-Research/AI-Research-SKILLs · MIT

Quick start

  1. 1Download and unzip the skill. Download the slime-rl-training ZIP from this page and unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder.
  2. 2Prepare your hardware and Docker. Confirm you have NVIDIA GPUs (4 to 8 per node in the examples) and Docker with GPU support, or a manual Megatron-LM and SGLang install.
  3. 3Gather your model and data. Get a model checkpoint in HuggingFace or Megatron format and a training dataset in JSONL with prompt and label fields.
  4. 4Ask for the task in plain language. Describe the model, GPU count and goal, for example a GRPO run for Qwen3-4B on one node with 4 GPUs.
  5. 5Review the commands. Read each launch command, check paths, save interval and rollout count before anything runs.
  6. 6Run a short test first. Start with fewer rollouts to confirm the setup works, then scale up to the full training run.
  7. 7Monitor and troubleshoot. Check TensorBoard reward curves and GPU use. If something fails, share the error with the agent and use the troubleshooting reference.

What is an agent skill, and what does this one do?

An agent skill is a folder of instructions and reference files that an AI agent (such as Claude) can load when a task matches it. The skill's main file, SKILL.md, tells the agent when to use it and how to proceed. This skill, slime-rl-training, covers LLM post-training with RL using slime, from the THUDM team.

When the skill is active, the agent knows how to set up Docker, choose a pre-configured model script, launch a GRPO training run, run asynchronous training, and build a multi-turn agentic training setup with a custom generate function. It also has reference files on the slime API and common troubleshooting issues.

  • Installation and Docker setup commands
  • GRPO training launch with model config selection
  • Asynchronous training with buffer and weight-sync parameters
  • Multi-turn training with a custom generate function
  • Argument reference for Megatron, SGLang and slime settings
  • Troubleshooting notes for SGLang crashes, router load imbalance and weight sync timeouts

Install the slime-rl-training skill

Download the skill as a ZIP from our page and unzip it. Put the resulting folder in one of these locations: ~/.claude/skills/ to make it available to you across projects, or a project's .claude/skills/ folder to limit it to that project.

You can also upload the ZIP in the Claude apps under Settings → Capabilities → Skills, if your plan and app version offer that option.

  • Personal install: unzip into ~/.claude/skills/
  • Project install: unzip into the project's .claude/skills/ folder
  • Claude apps: Settings → Capabilities → Skills, then upload the ZIP

Trigger the skill with natural requests

You do not need to name the skill. The skill's description says it applies when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling. Phrase your request around that task, and mention slime or GRPO if you want to be sure.

Give the agent concrete details such as model family, number of GPUs and where your data lives. The skill's steps depend on those choices.

  • "Using the slime skill, set up a GRPO run for Qwen3-4B on one node with 4 GPUs."
  • "Help me launch slime async training with an async buffer size of 4 on 8 GPUs."
  • "I want a multi-turn agent with tool calls in slime. Walk me through a custom generate function."
  • "My slime run keeps timing out during weight sync. What should I check?"

Prepare the requirements before you run anything

The skill is a how-to reference. It does not train a model or supply compute. You need Docker with NVIDIA GPU support (or Megatron-LM and SGLang installed manually), NVIDIA GPUs (the examples use 4 to 8 per node), and Python with ray, torch>=2.0.0, transformers>=4.40.0 and sglang-router>=0.2.3.

You also need a model checkpoint in HuggingFace or Megatron format, and a training dataset in JSONL format. Each data line holds a prompt and a label, for example {"prompt": "What is 2 + 2?", "label": "4"}.

  • Docker with NVIDIA GPU support, or a manual Megatron-LM and SGLang install
  • NVIDIA GPUs: examples assume 4 to 8 per node
  • Python packages: ray, torch>=2.0.0, transformers>=4.40.0, sglang-router>=0.2.3
  • A model checkpoint (HuggingFace or Megatron format)
  • A training dataset in JSONL format
docker pull slimerl/slime:latest docker run --rm --gpus all --ipc=host --shm-size=16g \ -it slimerl/slime:latest /bin/bash # Inside container cd /root/slime && pip install -e . --no-deps

Run a GRPO training job

The skill's standard workflow has four steps: prepare data, configure the model, launch training, and monitor. For model settings, the skill points to pre-configured scripts in scripts/models/, such as qwen3-4B.sh or glm4-9B.sh. You source the one that matches your model before launching.

The skill suggests checking TensorBoard output (tensorboard --logdir outputs/), confirming that reward curves rise, and watching GPU use across nodes. Because a launch starts a long job that uses significant GPU time, read every command before running it.

  • Step 1: prepare data in JSONL, optionally with chat-format prompts
  • Step 2: list and source a model script from scripts/models/
  • Step 3: launch train.py with your node and GPU counts
  • Step 4: monitor TensorBoard, rewards and GPU utilisation
source scripts/models/qwen3-4B.sh python train.py \ --actor-num-nodes 1 \ --actor-num-gpus-per-node 4 \ --rollout-num-gpus 4 \ --advantage-estimator grpo \ --use-kl-loss --kl-loss-coef 0.001 \ --rollout-batch-size 32 \ --n-samples-per-prompt 8 \ --global-batch-size 256 \ --num-rollout 3000 \ --prompt-data /path/to/data.jsonl \ ${MODEL_ARGS[@]} ${CKPT_ARGS[@]}

Use asynchronous and multi-turn modes

Asynchronous training overlaps rollout generation and training to raise throughput. The skill suggests it for large models with long generation times, or when GPUs sit idle in synchronous mode. It launches train_async.py with --async-buffer-size, and --update-weights-interval controls how often weights sync to rollout.

For agents that use tools, the skill shows a custom generate function that loops over turns, runs tool calls and computes a reward. You pass it with --custom-generate-function-path and cap turns with --max-turns. The source points to examples/search-r1/ as a full multi-turn search example.

  • Async: train_async.py, --async-buffer-size 4, --update-weights-interval 2
  • Multi-turn: --custom-generate-function-path custom_generate.py, --max-turns 5
  • Full example reference: examples/search-r1/
python train_async.py \ --actor-num-nodes 1 \ --actor-num-gpus-per-node 8 \ --rollout-num-gpus 8 \ --advantage-estimator grpo \ --async-buffer-size 4 \ --prompt-data /path/to/train.jsonl \ ${MODEL_ARGS[@]}

Safety notes before running commands

This skill runs scripts and uses the network: it includes commands that pull a Docker image, install packages, and launch training jobs. Review each command yourself before your agent runs it, especially on a shared or production cluster.

Training jobs can consume large amounts of GPU time and may write checkpoints to disk. Check your save interval and output paths first. Set the --save-interval and output locations deliberately instead of accepting defaults blindly.

  • Read commands before running them; the agent should show them first
  • Confirm paths for --prompt-data and checkpoints
  • Check --num-rollout and --save-interval against your time and disk budget
  • Use a test run with fewer rollouts before a long job

Tips for getting better results

Name your hardware and model in the first message. The skill's options change with GPU count and model family. If a run fails, paste the error into the conversation, since the troubleshooting reference covers SGLang crashes, router load imbalance and weight sync timeouts.

Keep the batch constraint in mind: rollout_batch_size × n_samples_per_prompt must equal global_batch_size × num_steps_per_rollout. The skill's example is 32 × 8 = 256 × 1.

  • Give GPU count, model and data path up front
  • Paste full error output when troubleshooting
  • Use the batch-size constraint to check your settings
  • Fix a memory or crash issue by trying --sglang-mem-fraction-static, a smaller --rollout-batch-size, or --use-fault-tolerance
rollout_batch_size × n_samples_per_prompt = global_batch_size × num_steps_per_rollout

What people use it for

  • Reproducing a GLM-style or Qwen3-style GRPO training run on a single multi-GPU node
  • Running asynchronous RL training on a large model where rollout generation is slow
  • Building a multi-turn agent that makes tool calls, with a custom generate function and reward
  • Using a custom data buffer to filter or generate prompts for an RL run
  • Checking an existing slime run for SGLang crashes, router imbalance or weight sync timeouts

Frequently asked questions

Is the slime RL Training skill free?

The skill is published under the MIT licence, as listed on this page. Running it still costs you compute, since slime needs NVIDIA GPUs and the examples run long training jobs.

What hardware do I need to use this skill?

You need NVIDIA GPUs, with the examples assuming 4 to 8 per node, plus Docker with GPU support or a manual Megatron-LM and SGLang install. The skill cannot provide compute, so beginners without multi-GPU access will not be able to run the examples.

How do I install an agent skill ZIP?

Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use, or into a project's .claude/skills/ folder. In the Claude apps you can also upload it under Settings → Capabilities → Skills.

Does the skill train a model by itself?

No. It is a how-to reference that gives the agent commands and guidance. You still need your own GPUs, model checkpoint and JSONL dataset before any training starts.

What is the difference between slime and verl?

The skill's guidance says to choose slime for Megatron-LM native training with SGLang inference and flexible data buffers, and to consider verl if you want flexible backend swapping. Pick based on whether you already use Megatron-LM.

Why does my slime training crash or time out?

The skill's troubleshooting reference covers SGLang engine crashes, router load imbalance and weight sync timeouts. Suggested fixes include --use-fault-tolerance, a higher --sglang-mem-fraction-static, a smaller --rollout-batch-size, and --sglang-router-strategy round_robin for imbalance.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Use this skill if you are an ML engineer or researcher with multi-GPU NVIDIA hardware who wants an agent to help set up slime GRPO, async or multi-turn RL runs with Megatron-LM and SGLang. Pick something lighter if you only need simple fine-tuning or inference, or if you lack the GPUs to run the examples.

Download the skill