Guide · Agent skill

How to use the Modal Cloud GPU Runner agent skill

Updated 2026-10-29 · written with AI from the project's own documentation and checked against our catalogue data

The serverless-modal skill lets an AI coding agent write Python launcher scripts that run your code on remote GPUs rented from Modal, a serverless cloud provider, instead of on your own machine. It is aimed at developers without a local GPU and researchers who want to launch a training, fine-tuning, inference or benchmark job without managing servers or Docker. After this guide you will know how to install the skill, set up a Modal account safely, trigger the skill with a plain request, and review the cost estimate it shows you before any run starts.

Modal Cloud GPU Runnerwanshuiyin/Auto-claude-code-research-in-sleep · MIT Download the skill

Quick start

1Download and install the skill. Download the ZIP from this page and unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder, or upload it under Settings → Capabilities → Skills in the Claude apps.
2Install the Modal CLI and log in. Run pip install modal, then modal setup to log in through your browser.
3Set a spending limit. On modal.com, bind a card only if you want the larger free tier, then set a workspace spending limit under Settings → Usage → Spending Limit.
4Make a request. Ask the agent for a GPU task in plain language, such as "run this training script on a remote GPU using Modal".
5Review the cost estimate. Check the model, VRAM, GPU choice, duration and dollar estimate the agent shows you, and confirm only if it matches your expectations.
6Run or deploy. Let the agent run modal run launcher.py for a one-shot job or modal deploy app.py for an inference endpoint.
7Check the results. Look in the Modal Volume named in your launcher, such as experiment-results, for saved outputs.

What the skill makes the agent do

Without the skill, an agent edits code on your laptop and runs it there. With the skill installed, the agent can write a Python launcher that uses Modal's CLI and Python library to send a GPU function to Modal's infrastructure. Your code, data and results stay local; only the GPU function runs remotely.

Modal is a serverless GPU cloud. You do not SSH into a machine or manage Docker or port forwarding, and billing stops when your code finishes, so there are no idle instances left running. The skill also suggests a GPU based on model size and VRAM needs, and it guides login and spending-limit setup.

Writes a launcher script for a one-shot GPU job (training, evaluation or benchmark) or a persistent web API for inference.
Declares dependencies in code with modal.Image so environments can be reproduced.
Estimates cost and shows it to you for confirmation before every run.

Download the skill ZIP from this page. The skill is a single file, SKILL.md. Install it in one of two ways.

Option 1, personal skills folder: unzip the archive into ~/.claude/skills/ so the skill is available in all your projects. Option 2, project folder: unzip it into the .claude/skills/ folder of a specific project so it applies only there.

Option 3, Claude apps: upload the ZIP in the Claude apps under Settings → Capabilities → Skills.

Personal: ~/.claude/skills/
Project: .claude/skills/ inside your project folder
Claude apps: Settings → Capabilities → Skills

The skill needs Python with pip, the Modal CLI and a Modal account. Install the CLI, then log in. The login opens a browser and writes a token to ~/.modal.toml.

Sign up at modal.com with GitHub or Google. The free tier without a card gives $5 per month, which is enough for quick tests. Binding a payment card on modal.com unlocks $30 per month. Do this only on the official website, never by typing card details into the agent or any CLI tool.

Set a workspace spending limit under Settings → Usage → Spending Limit (for example $30) so you cannot exceed the free tier by accident. Modal pauses your workloads when the limit is hit.

Requirements: Python with pip, the Modal CLI, a Modal account.
Optional: a Hugging Face token or other secrets, stored with modal secret create.
Optional: a payment card, entered only on modal.com.

pip install modal
modal setup
modal run -q 'print("ok")'

Trigger the skill with a natural request

The skill activates when your request matches its description. You do not need to name the skill. Phrases it responds to include "modal run", "modal training", "modal inference", "deploy to modal", "need a GPU", "run on modal" and "serverless GPU", or any request for remote GPU compute.

Describe the task and the model size where you can. The more specific you are, the better the GPU suggestion. The skill also accepts a task description as an argument.

"Fine-tune a 7B model on my dataset on a remote GPU using Modal."
"I need a GPU to debug this CUDA kernel, run on modal."
"Deploy a small inference endpoint to modal for my 1B model."

Review the cost estimate before approving a run

Before every run, the agent must estimate the cost and show it to you. The estimate lists the model, its VRAM need, the chosen GPU with its hourly and per-second price and memory bandwidth, and the expected duration and total cost. Do not approve a run until the numbers match what you expect.

The skill gives rules of thumb for VRAM: a 7-8B model in FP16 needs about 22GB, so a 16GB T4 is not enough, and an L4, A10 or A100-40GB fits. Inference is usually limited by memory bandwidth rather than compute, so a higher-bandwidth GPU can be cheaper overall. Pricing is per second, and the GPU is typically over 90% of total cost.

Model, parameter count and precision
VRAM needed (weights, KV cache and overhead)
GPU type, price per second and per hour, bandwidth
Estimated minutes and dollars

Cost estimate (Modal):
  Model: [name] ([params], [precision])
  VRAM: ~[X]GB (weights + KV cache + overhead)
  GPU: [type] ([VRAM]GB, $[X]/sec = $[X]/hr, bandwidth [X] GB/s)
  Estimate: ~[N] min, ~$[X]

Run a one-shot training job or deploy an inference API

For training, evaluation or benchmarks, the skill writes a launcher with a @app.function that runs your script on a chosen GPU, mounts your local project folder and saves results to a Modal Volume. Run it with modal run launcher.py. The example sets a six-hour timeout and calls volume.commit() so results persist.

For inference, the skill writes a class that loads the model once and exposes a POST endpoint with @modal.fastapi_endpoint. Deploy it with modal deploy app.py. Use the output to check the GPU and model before scaling up.

One-shot: modal run launcher.py
Persistent web API: modal deploy app.py

modal run launcher.py
modal deploy app.py

Keep secrets and dependencies safe and reproducible

Store API tokens, such as a Hugging Face token, as Modal secrets rather than pasting them into code. The skill attaches a named secret to the function that needs it.

Dependencies are declared in the modal.Image chain. Each pip_install call installs in order, so you can pin torch in one call and install other packages in a later one without the later packages changing torch.

Create a secret: modal secret create huggingface-secret HF_TOKEN=hf_xxxxx (replace with your own token)
Attach it in code with modal.Secret.from_name

modal secret create huggingface-secret HF_TOKEN=hf_xxxxx

What people use it for

Debugging CUDA code on a GPU when your laptop has none.
Running a short fine-tuning or evaluation job on an A100 or L40S and paying only for the seconds used.
Benchmarking inference speed across GPU types to compare cost per 1000 samples.
Deploying a small model behind a POST endpoint for quick testing.
Iterating on experiments that each take under a few hours.

Frequently asked questions

How do I install the serverless-modal skill?

Download the ZIP from this page and unzip it into ~/.claude/skills/ for all projects, or into a project's .claude/skills/ folder for one project. You can also upload the ZIP in the Claude apps under Settings → Capabilities → Skills.

What do I need to use the Modal GPU skill?

You need Python with pip, the Modal CLI installed with pip install modal, and a Modal account. A Hugging Face token or other secrets are optional, and a payment card on modal.com is optional for the larger free tier.

Is Modal free to use with this skill?

Modal has a free tier of $5 per month without a card and $30 per month with a payment card bound on modal.com. Paid GPU usage beyond the free allowance is billed per second. Set a workspace spending limit to avoid unexpected charges.

Does the skill send my code or data to the cloud?

Your code, data and results stay on your machine, but the GPU function runs on Modal's infrastructure, so the code it executes is sent there. If you must keep code or data off third-party clouds, do not use this skill.

Is Modal cheaper than other GPU providers?

It depends on the job. Modal costs more per GPU-hour than vast.ai or Lightning for some tiers, but it has no setup time and no idle billing, which can make short and medium jobs cheaper. For long training runs over about four hours, the skill itself suggests a provider with a lower hourly rate such as vast.ai.

Why does the skill ask me to approve a cost estimate?

The skill is designed to estimate cost before every run and show it to you for confirmation. This lets you catch a GPU choice or duration that would cost more than you expect before any money is spent.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Use this skill if you need a GPU without owning one, want to launch short or medium experiments without managing servers, and are comfortable with your code running on Modal's infrastructure. Pick a different option for multi-hour training runs, where the skill itself points to cheaper providers, or if your data must never leave your own machines.

Download the skill