Ray Train Distributed Training
Scale PyTorch, TensorFlow or HuggingFace training across one or many machines with Ray
- Repository
- Orchestra-Research/AI-Research-SKILLs
- Stars
- 14,195
- Licence
- MIT
- Files
- 2 files
What it does
This skill teaches the agent how to use Ray Train, an open-source library for running machine learning training jobs across several GPUs or machines at once. Once installed, the agent can take a normal single-machine PyTorch or HuggingFace training script, wrap it in Ray's trainer classes, and help run it on a cluster. It also covers hyperparameter sweeps with Ray Tune, checkpointing so failed runs can resume, and multi-node setup.
Great for
- ML engineers who need to train large models across multiple GPUs or machines
- Researchers running large hyperparameter sweeps who want distributed trials
- Teams that already run or plan to run a Ray cluster
Skip it if
- Small models that train fine on one GPU, where the added cluster setup adds complexity without benefit
- Anyone without access to GPUs or a cluster, since the examples assume GPU hardware and a Ray environment
- Production training of sensitive data without review, because the generated code launches real compute jobs that can incur cloud costs
Try asking
- “Convert my single-GPU PyTorch training loop so it runs on 8 GPUs with Ray Train”
- “Set up a Ray Tune sweep over learning rate and batch size with 20 trials and early stopping”
- “Add checkpointing to my training job so it resumes after a node failure”
- “Help me connect to an existing Ray cluster and launch a 2-node HuggingFace training run”
Install in 3 steps
- 1
Download the skill
The ZIP contains the 2 original files, the MIT licence and an attribution note.
ray-train.zip - 2
Add it to your agent
Claude Code (all projects)
mkdir -p ~/.claude/skills && unzip ray-train.zip -d ~/.claude/skills/Claude Code / Agent SDK (this project only)
mkdir -p .claude/skills && unzip ray-train.zip -d .claude/skills/Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.
- 3
Ask for the task
No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.
Get better results
- Review any generated code before running it, especially cluster addresses and GPU counts, since these determine cost and resource use
- Start with a small num_workers value on a single node to confirm the wrapped training loop works before scaling out
- Ask the agent to explain which Ray version and cluster setup it assumes, because the examples are illustrative and may need adjusting for your environment
What's inside
- references/multi-node.md
- SKILL.md
Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.