Guide · Agent skill
How to use the distributed-training agent skill for PyTorch DDP
Updated · written with AI from the project's own documentation and checked against our catalogue data
The distributed-training skill is a short set of best-practice instructions that tells an AI agent how to handle training a machine learning model across several GPUs at once, using PyTorch's DistributedDataParallel (DDP). It is written for ML engineers moving a PyTorch job from one GPU to several, researchers who want an agent to write or review multi-GPU code, and developers new to DDP. After following this guide you will know how to install the skill, how to ask for it in plain language, and what guidance the agent will apply.
Distributed Training Guideaiming-lab/AutoResearchClaw · MIT Download the skill Quick start
- 1Download the ZIP. Download the distributed-training skill ZIP from this page.
- 2Unzip into a skills folder. Unzip it into ~/.claude/skills/ for all projects, or into a project's .claude/skills/ folder for one project.
- 3Or upload in the Claude apps. In the Claude apps, go to Settings → Capabilities → Skills and upload the skill.
- 4Make a request. Ask for help with multi-GPU or distributed training in PyTorch, using words like DDP, multi-GPU or scale.
- 5Share your code. Paste your training script and the number of GPUs you plan to use.
- 6Review the changes. Read each suggested change, check the rank 0 checkpoint rule and learning rate update, and test on your setup.
What is an agent skill and what does this one do?
An agent skill is a folder containing a SKILL.md file with instructions that an AI agent reads when a task matches the skill's description. This skill is a checklist, not a program. Once it is installed, the agent prefers DDP over the older DataParallel, initializes the process group with the NCCL backend, shards data with DistributedSampler, converts batch norm to synchronized batch norm, saves checkpoints only on rank 0, scales the learning rate with the number of GPUs, and uses gradient accumulation for larger effective batch sizes.
The skill is guidance only. It does not run anything by itself, and the agent still writes or reviews the code you ask for.
Install the distributed-training skill
Download the skill ZIP from this page. Then choose one of the two installation routes below.
For Claude Code, unzip the file into your personal skills folder ~/.claude/skills/ or into a project's .claude/skills/ folder. Use the personal folder if you want the skill in every project, and the project folder if you want it only for one codebase.
In the Claude apps, open Settings, go to Capabilities, and upload the skill under Skills.
- Personal install: ~/.claude/skills/
- Project install: .claude/skills/ inside your project
- Claude apps: Settings → Capabilities → Skills
Trigger the skill with natural requests
You do not need a special command. The skill's description is about multi-GPU and distributed training with PyTorch DDP, and its trigger keywords include distributed, multi-gpu, parallel, ddp and scale. Mention one of these ideas in your request and the agent can pick the skill up.
Be specific about your framework and goal. The skill only covers PyTorch DDP, so say PyTorch when you ask.
- Help me move this PyTorch training script from one GPU to four using DDP.
- Review my training loop for multi-GPU problems, especially checkpoint saving.
- Scale this job to 8 GPUs and tell me how to change the learning rate.
Examples of what the agent will apply
Suppose you ask the agent to adapt a single-GPU PyTorch script. Guided by the skill, it should replace DataParallel with DistributedDataParallel, call dist.init_process_group(backend='nccl'), wrap the dataset with DistributedSampler, convert the model with nn.SyncBatchNorm.convert_sync_batchnorm(), and restrict checkpoint saving to rank 0.
For a review task, the agent can check an existing script against the same seven practices and point out where one is missing, such as a checkpoint written by every process or a learning rate that was not adjusted for the GPU count.
dist.init_process_group(backend='nccl')What you need before you start
The skill assumes you have PyTorch and that its DDP API is available. The patterns only matter when you have multiple GPUs and NCCL support. The skill does not cover setup of the cluster or machines.
The skill is not useful for single-GPU or CPU-only projects, for TensorFlow or JAX, or as the sole source for production decisions.
- PyTorch installed
- Multiple GPUs with NCCL support
- A training script to adapt or review
Safety notes: scripts and network
This skill runs no scripts and uses no network access, according to its listing. It consists of a single SKILL.md file. Because it is only instructions, the risk comes from the code the agent writes, so review any generated training code before running it on shared or expensive hardware.
Check the generated code for the rank 0 checkpoint rule and the learning rate change, since the checklist is brief and does not cover edge cases such as limits of linear learning-rate scaling or cluster configuration.
Tips for getting the best results
Give the agent your current script and the number of GPUs. The checklist is generic, so concrete code lets the agent apply each rule to the right lines.
Treat the skill as a reminder of standard steps. Verify behavior on your own setup and consult expert review for production runs.
- Paste the training loop, not just a description.
- State the GPU count so learning rate and batch size guidance can be applied.
- Ask the agent to explain each change so you learn the DDP pattern.
What people use it for
- Converting a single-GPU PyTorch training script to DistributedDataParallel.
- Reviewing an existing multi-GPU training loop for missing DistributedSampler or unsynchronized batch norm.
- Checking that checkpoints are saved only from the main process.
- Adjusting learning rate and gradient accumulation when moving to more GPUs.
Frequently asked questions
Is the distributed-training skill free?
The skill is published under the MIT licence in the aiming-lab/AutoResearchClaw repository. The source does not mention pricing, so check the listing for any terms about the platform you use with it.
What do I need to use the distributed-training skill?
You need PyTorch, since the skill assumes its DDP API, and multiple GPUs with NCCL support for the patterns to matter. It is not useful for single-GPU or CPU-only projects.
Does the skill run code or access the network?
According to its listing, it runs no scripts and uses no network. It is a SKILL.md file of guidance that the agent reads.
Does it work with TensorFlow or JAX?
No. The content only covers PyTorch DistributedDataParallel, so use a different resource for TensorFlow or JAX.
What is the difference between DDP and DataParallel?
The skill tells the agent to prefer DistributedDataParallel over the older DataParallel for multi-GPU training. The source does not go into further detail on the differences.
Where do I put the skill folder after unzipping?
Place it in ~/.claude/skills/ for personal use across projects, or in .claude/skills/ inside a specific project. In the Claude apps, upload it under Settings → Capabilities → Skills.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you are scaling a PyTorch job across GPUs, or you want an agent to write or review DDP code and remind you of the standard steps. Pick something else if you use single-GPU or CPU-only setups, TensorFlow or JAX, or need a thorough guide for production clusters, since this checklist is brief and does not cover those edge cases.
Download the skill