Skip to content
Get packs

Guide · Agent skill

How to use the Ray Train distributed training agent skill

Updated · written with AI from the project's own documentation and checked against our catalogue data

Ray Train is an open-source library for running machine learning training across several GPUs or machines at once. This agent skill teaches an AI agent how to take a normal PyTorch or HuggingFace training script, wrap it in Ray's trainer classes and help you run it on a cluster. After this guide you will know how to install the skill, how to phrase a request that triggers it, and what you need before launching any compute job.

Ray Train Distributed TrainingOrchestra-Research/AI-Research-SKILLs · MIT

Quick start

  1. 1Download and unzip the skill. Download the ZIP from this page and unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder. In the Claude apps you can instead upload it under Settings, Capabilities, Skills.
  2. 2Install Ray Train. Run pip install -U "ray[train]" in the Python environment you will train in. You also need PyTorch, and transformers if you use the HuggingFace workflow.
  3. 3Describe your training script. Ask the agent to convert your existing training code into a Ray Train version, naming the framework and the number of GPUs you have.
  4. 4Review the scaling settings. Check num_workers, use_gpu and any resources_per_worker values in the generated code before running it, since they determine how much compute is used.
  5. 5Start your cluster for multi-node runs. For more than one machine, start the head node with ray start --head and connect each worker with ray start --address pointing at the head node. Then run the training script.

What does the ray-train skill make the agent do?

Once installed, the skill gives the agent working knowledge of Ray Train. The source describes it as covering normal single-machine training code, hyperparameter sweeps with Ray Tune, checkpointing so failed runs can resume, and multi-node setup.

  • Wrap a single-machine PyTorch training function in a TorchTrainer and set the number of workers and whether they use GPUs.
  • Wrap a HuggingFace Transformers training setup in a TransformersTrainer.
  • Run distributed hyperparameter sweeps with Ray Tune, including an ASHAScheduler for early stopping.
  • Add checkpoints with train.report so a run can resume from its last saved state.
  • Explain how to start a Ray head node and connect worker nodes.

How do you install the ray-train skill?

Download the ZIP for the skill from this page. Unzip it into ~/.claude/skills/ to make it available in all your projects, or into a project's .claude/skills/ folder to limit it to that project.

If you use the Claude apps instead, open Settings, go to Capabilities, then Skills, and upload the skill there.

The skill itself does not install Ray. You still need Python and the Ray Train package, which the source installs with the command below.

pip install -U "ray[train]"

How do you trigger the skill with a natural request?

You do not need a special command. The agent is expected to use the skill when your request matches its description: training across multiple machines, running large models that need several GPUs, or running distributed hyperparameter sweeps. Name the framework and the goal so the agent knows which parts to use.

The source does not list exact trigger phrases, so treat these as examples of requests that fit the skill's description rather than guaranteed wording.

  • Convert this single-GPU PyTorch training script so it runs on 4 GPUs with Ray Train.
  • Set up a Ray Tune sweep over learning rate and batch size for this model and report the best config.
  • Add checkpointing so training resumes from the last checkpoint if a worker fails.
  • Write a Ray Train launch for a 2-node cluster with 8 GPUs per node.

How do you set up a multi-node Ray cluster?

Multi-node training needs a Ray cluster with one head node and one or more worker nodes. The head node coordinates workers and runs scheduling, and the worker nodes execute training tasks. The reference file in the skill shows the head node started with a dashboard bound to all interfaces.

On the head node, start Ray. On each worker node, connect it to the head node's address. Replace the placeholder with your head node's IP address. Your training script then connects with ray.init(address='auto').

  • The reference example uses port 6379 and a dashboard flag, --dashboard-host=0.0.0.0. Check your network settings before exposing a dashboard.
  • The source's example output lists a dashboard at port 8265 on the head node.
ray start --head --port=6379 ray start --address=<head-node-ip>:6379

What are the safety, cost and network notes?

The skill uses the network and the listing states it does not run scripts on its own. The code it produces, however, launches real compute jobs on your GPUs or cluster, and those jobs can incur cloud costs. Read generated code before running it, especially the number of workers.

Do not use the generated code for production training on sensitive data without a review. The examples assume GPU hardware and a working Ray environment, so they will not run on a machine without them.

  • Check num_workers and use_gpu in ScalingConfig before every run.
  • Start with a small number of workers to confirm the training loop works.
  • Keep a copy of your original single-machine script so you can compare results.
  • Confirm which cloud account and billing will be charged for any cluster you start.

Tips for getting better results

The skill's own guidance is that multi-node setup, tuning at scale, fault tolerance and elastic scaling are the reasons to use Ray Train. If your model trains fine on one GPU, a single-machine run is simpler. Ray Train also works with PyTorch, TensorFlow and HuggingFace, so state which one your project uses.

  • Use SPREAD placement when you want workers spread across nodes.
  • Save checkpoints on a regular schedule so a failed run has something to resume from.
  • Use Ray Tune only when you actually need a sweep; a single run is cheaper to debug.

What people use it for

  • Scaling a single-GPU PyTorch training loop to several GPUs on one machine.
  • Training a large HuggingFace Transformers model across multiple nodes with a Ray cluster.
  • Running a learning-rate and batch-size sweep with Ray Tune and early stopping.
  • Adding checkpoints so long multi-node training runs can resume after a worker failure.

Frequently asked questions

How do I install the ray-train agent skill?

Download the ZIP from this page and unzip it into ~/.claude/skills/ for all projects, or into a project's .claude/skills/ folder for one project. In the Claude apps, upload it under Settings, Capabilities, Skills. You also need to install Ray Train with pip install -U "ray[train]".

Is the ray-train skill free?

The skill is listed under the MIT licence. Ray Train is an open-source library. Any cloud GPUs or cluster compute you launch with the generated code are billed by your provider, and the source does not give prices.

What do I need to use the ray-train skill?

You need Python, the ray[train] pip package, and PyTorch. The HuggingFace workflow also needs transformers. The GPU examples need GPU hardware, and multi-node training needs a running Ray cluster.

Does the ray-train skill run scripts on my computer?

The listing says the skill does not run scripts and does use the network. It teaches the agent to write Ray Train code, but that code launches real compute jobs when you run it, so review it first.

Why does my Ray Train job fail or not reach my cluster?

The source does not list specific error messages. Check that the head node is running with ray start --head, that each worker was connected with the correct head node address and port, and that your machines have the GPUs the script requests.

When should I not use Ray Train?

The listing says not to use it for small models that train fine on one GPU, because the cluster setup adds complexity without benefit. It also says the source does not compare Ray Train with other tools, so check those yourself before choosing.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Use the ray-train skill if you train large models across several GPUs or machines, run big hyperparameter sweeps, or already run a Ray cluster. Pick something else if your model fits on one GPU, you have no GPU or cluster access, or you need to train on sensitive data without a careful review of the generated code and its costs.

Download the skill