Skip to content
Get packs

Guide · Agent skill

How to use the Run ML Experiments agent skill (run-experiment)

Updated · written with AI from the project's own documentation and checked against our catalogue data

The run-experiment skill is a set of instructions that lets an AI coding agent launch machine learning training jobs for you on a GPU, whether that is a remote server reached over SSH, a rented Vast.ai instance, Modal serverless GPUs, or your own local machine. It is written for ML researchers and small teams who already have GPU access and want the agent to follow the same launch routine every time. This guide explains what the skill does, how to install it, how to trigger it, and what to check before you let it run commands.

Run ML Experimentswanshuiyin/Auto-claude-code-research-in-sleep · MIT

Quick start

  1. 1Install the skill. Download the ZIP from this page and unzip it into ~/.claude/skills/ or the project's .claude/skills/ folder, or upload it under Settings → Capabilities → Skills in the Claude apps.
  2. 2Write your CLAUDE.md. Add the gpu setting and the matching details: SSH alias, conda env and code directory for remote, or a ## Vast.ai or ## Modal section.
  3. 3Confirm prerequisites. Make sure you have SSH access, NVIDIA drivers with nvidia-smi, and screen and rsync on the target machine.
  4. 4Make a natural request. Ask something like "run experiment" or "deploy to server" and describe the experiment in one line.
  5. 5Let the pre-flight check run. The agent checks GPU memory use. A GPU with under 500 MiB used counts as free.
  6. 6Review the sync and launch commands. Read the rsync or git command and the screen launch line before approving, especially on shared machines.
  7. 7Check the log file. Follow the job output in the log file the skill wrote, and check the screen session for the named experiment.

What is an agent skill and what does run-experiment do?

An agent skill is a folder containing a SKILL.md file with instructions that an agent such as Claude loads when a request matches the skill's description. The run-experiment skill's SKILL.md tells the agent how to deploy and run an ML experiment step by step.

The routine it describes: read your project's CLAUDE.md to find where the GPU is, check that the GPU is free, copy over only the code files, optionally add Weights & Biases logging, and start the training job in a named screen session with its output saved to a log file. For Vast.ai and Modal it hands off to the /vast-gpu and /serverless-modal skills.

  • Reads the gpu setting in CLAUDE.md (local, remote, vast or modal)
  • Checks GPU memory use with nvidia-smi and treats a GPU with under 500 MiB used as free
  • Syncs only code files by default using rsync, or uses git if code_sync: git is set
  • Adds Weights & Biases logging when wandb: true is set in CLAUDE.md
  • Launches the job in a dedicated screen session with output written to a log file

Install the run-experiment skill

Download the ZIP from this page and unzip it. The skill folder contains SKILL.md. For a personal install, place the folder in ~/.claude/skills/. For a single project, place it in that project's .claude/skills/ folder.

You can also upload the skill in the Claude apps: open Settings, go to Capabilities, then Skills, and upload the ZIP there.

  • Personal: ~/.claude/skills/ (available across your projects)
  • Project only: .claude/skills/ inside the project folder
  • Claude apps: Settings → Capabilities → Skills, then upload the ZIP

Prepare your CLAUDE.md file

The skill finds your environment by reading CLAUDE.md at the project root. Without it, the agent has to ask you where to run the job. Describe the GPU mode and the details for that mode.

For a remote server, include the SSH alias, conda environment and code directory. For Vast.ai, add a ## Vast.ai section. For Modal, add gpu: modal or a ## Modal section. Optional settings include code_sync, wandb and the modal_ settings.

  • gpu: local, remote, vast or modal
  • Remote: SSH alias, conda env, code directory
  • code_sync: git (otherwise rsync is the default)
  • wandb: true, with the W&B project name and API key
  • Modal: modal_gpu, modal_timeout (default 21600 seconds), modal_volume

Trigger the skill with natural requests

You do not need a special command. Phrases such as "run experiment", "deploy to server", "跑实验" or a request to launch training jobs match the skill's description. You can add a short description of the experiment after the request, since the skill's argument hint is [experiment-description].

Be specific about the experiment and, if relevant, which GPU or provider you want. The skill will still read CLAUDE.md for the server details.

  • "Run experiment: train the baseline on the 2 GPU server with the new learning rate schedule"
  • "Deploy to server and start the ablation sweep"
  • "Launch the training job on Vast.ai"

Understand what the agent does during a launch

The skill's steps run in order: detect the environment, run a pre-flight GPU check, sync code, optionally add W&B logging, then deploy. For remote and Vast.ai runs, the job is started with screen -dmS so it keeps running after the SSH session ends, and output is piped through tee into a log file.

Remote and local runs use CUDA_VISIBLE_DEVICES to select the GPU. On a Mac with MPS, the skill checks torch.backends.mps.is_available() instead of nvidia-smi.

  • Pre-flight: nvidia-smi query for memory used and total per GPU
  • Sync: rsync excludes data/, checkpoints (*.pt, *.pth, *.ckpt), wandb/ and outputs/ for Vast.ai
  • Launch: named screen session per experiment, activated conda env on remote machines
ssh <server> nvidia-smi --query-gpu=index,memory.used,memory.total --format=csv,noheader

Check the requirements before you start

The skill only automates launching jobs. It does not supply a GPU. You need an existing GPU machine or a cloud GPU account. The source lists these requirements.

  • SSH access to the target GPU server or Vast.ai instance
  • NVIDIA drivers with nvidia-smi (or PyTorch for Mac MPS checks)
  • screen and rsync installed on the target machine
  • A project CLAUDE.md describing the environment
  • Optional: a Weights & Biases account and API key if wandb logging is enabled
  • Optional: the Vast.ai CLI (vastai) and a Modal account, used through /vast-gpu and /serverless-modal

Safety notes: scripts, network and permissions

This skill runs scripts and uses the network. Its allowed-tools setting is Bash(*), which gives the agent broad shell access, along with Read, Grep, Glob, Edit, Write and the serverless-modal skill. It runs shell commands, rsync and package installs on the machines you point it at.

Do not point it at production servers or shared machines unless you are willing to review each command first. Check the target in CLAUDE.md before approving a launch, and confirm the SSH alias points where you expect.

  • Review commands before approving them, especially on shared machines
  • Confirm the SSH alias, host and port match the machine you intend to use
  • Remember that rsync and pip install run on the target
  • Keep wandb API keys out of shared files where possible

What people use it for

  • Starting a training run on a lab GPU server you reach over SSH, with the job in its own named screen session
  • Launching a fine-tuning job on a rented Vast.ai instance without writing the setup commands yourself
  • Running an experiment on Modal serverless GPUs with the launcher generated for you
  • Adding Weights & Biases metric logging to a training script before a run
  • Keeping consistent launches across a team by sharing one CLAUDE.md layout

Frequently asked questions

Is the run-experiment skill free to use?

The skill is published under the MIT licence, as stated on this page. Cloud compute is separate: Vast.ai and Modal are paid services that need your own account, and this page does not list their prices.

What do I need before I can use run-experiment?

You need SSH access to a GPU server or Vast.ai instance, NVIDIA drivers with nvidia-smi (or PyTorch for Mac MPS), screen and rsync on the target machine, and a CLAUDE.md file describing your environment. The skill does not provide compute.

Where do I put the run-experiment folder?

Place it in ~/.claude/skills/ for personal use across projects, or in .claude/skills/ inside a single project. You can also upload the ZIP in the Claude apps under Settings → Capabilities → Skills.

Why won't the skill launch my job?

The most likely causes are a missing or incomplete CLAUDE.md, no SSH access to the target, or screen and rsync not installed on it. Also check that the GPU has more than 500 MiB in use, since the skill treats a GPU with less than that as free and a busier one as occupied.

Does run-experiment work for things other than ML training?

No. The skill is specific to training and experiment deployment, so it is not suited to other kinds of tasks.

How does run-experiment compare with launching jobs by hand?

The skill follows a fixed routine so launches are consistent, and it covers the pre-flight check, selective code sync and screen session setup. Doing it manually gives you more control per command, which is worth it for shared or production machines.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Use run-experiment if you train models on a GPU server you can reach over SSH, or rent GPUs on Vast.ai or Modal, and want the agent to follow the same launch steps each time from your CLAUDE.md settings. Skip it if you have no GPU machine or cloud account, if your task is not ML training, or if you want to run an agent against shared or production servers without reviewing every command, since the skill runs shell commands with broad Bash(*) permissions.

Download the skill