Guide · Agent skill
How to use the data-loading agent skill for PyTorch pipelines
Updated · written with AI from the project's own documentation and checked against our catalogue data
The data-loading skill is a short set of written guidelines that an AI agent reads before it writes or reviews data-feeding code for PyTorch training. Its aim is to stop a GPU from sitting idle while it waits for batches to arrive. This guide explains what an agent skill is, how to install this one, how to ask for it, and what to check before you trust the result.
Data Loading Optimizationaiming-lab/AutoResearchClaw · MIT Download the skill Quick start
- 1Download the ZIP. Download the data-loading skill ZIP from this page. It contains the SKILL.md file.
- 2Unzip into a skills folder. Place the data-loading folder in ~/.claude/skills/ for personal use or in a project's .claude/skills/ folder.
- 3Or upload it in the Claude apps. In the Claude apps, open Settings → Capabilities → Skills and upload the ZIP.
- 4Confirm PyTorch is available. The advice refers to DataLoader and torchvision.transforms.v2, so make sure your project uses PyTorch.
- 5Describe your data pipeline. Ask for help setting up a DataLoader or reviewing preprocessing, and mention any idle GPU symptoms.
- 6Review the suggested changes. Check each recommended argument against your hardware before applying it.
- 7Measure the result. Profile with torch.utils.bottleneck and compare epoch times, since the file gives rules of thumb rather than benchmarks.
What an agent skill is and what this one does
An agent skill is a folder containing a SKILL.md file. The file holds instructions that an agent such as Claude loads when a task matches the skill's description. This skill is one of those folders, and its file is named SKILL.md.
Its description reads: optimize data loading pipeline to prevent GPU starvation, used when setting up DataLoader or data preprocessing. When it applies, the agent has a checklist of practices to follow. The checklist covers choosing a worker count, pinning memory when a GPU is used, keeping workers alive between passes, caching preprocessing, using torchvision.transforms.v2 for image data, considering memory-mapped files or WebDataset for large datasets, and profiling with torch.utils.bottleneck.
Install the data-loading skill
Download the ZIP for this skill from this page. It contains one file, SKILL.md. Unzip it so that the data-loading folder containing SKILL.md sits in one of the locations below.
For a personal setup, place the folder in ~/.claude/skills/. To share it with one project, place it in that project's .claude/skills/ folder. You can also upload the ZIP in the Claude apps under Settings → Capabilities → Skills.
- Personal: ~/.claude/skills/data-loading/
- Single project: .claude/skills/data-loading/ inside the project folder
- Claude apps: Settings → Capabilities → Skills, then upload the ZIP
Trigger the skill with natural requests
You do not need a special command. The skill is meant to load when your request involves DataLoader, datasets, preprocessing or augmentation. Its trigger keywords include data, loading, dataloader, dataset, preprocessing and augmentation.
Name the situation in plain words. Mention the symptom, such as a GPU that is idle or slow epochs, so the agent knows why you are asking.
- Set up a DataLoader for my image dataset and keep the GPU busy.
- Review this preprocessing script for slow input and tell me what to change.
- My training is slow and nvidia-smi shows low GPU use. Check the data pipeline.
What the agent will recommend
The skill gives the agent seven rules of thumb. Ask it to apply them to your code and explain each change.
Worker count: the skill suggests num_workers = min(8, os.cpu_count()). Pinned memory: pin_memory=True when a GPU is used. Persistent workers: persistent_workers=True to avoid re-spawning. Caching: pre-compute and cache transformations when possible. Image data: use torchvision.transforms.v2. Large datasets: consider memory-mapped files or WebDataset. Profiling: use torch.utils.bottleneck to find I/O bottlenecks.
- num_workers = min(8, os.cpu_count())
- pin_memory=True when using a GPU
- persistent_workers=True
- Pre-compute and cache transformations when possible
- Use torchvision.transforms.v2 for image data
- Consider memory-mapped files or WebDataset for large datasets
- Profile with torch.utils.bottleneck
num_workers = min(8, os.cpu_count())
DataLoader(dataset, num_workers=num_workers, pin_memory=True, persistent_workers=True)Safety notes: scripts and network
This skill does not run scripts and does not use the network. It is text guidance only. The agent still writes and runs code in your environment as it normally would, so review any changes before you run training.
The guidance is rules of thumb, not benchmarks. The file gives no measured speedups, so test the agent's changes on your own hardware and dataset.
- Runs scripts: no
- Uses network: no
- Requires PyTorch, since the advice refers to DataLoader and torchvision.transforms.v2
Tips for getting useful results
Give the agent your hardware details and the shape of your data. A worker count that suits an 8-core machine may not suit a shared server, and the rules only make sense with that context.
Ask the agent to profile before and after. The skill points to torch.utils.bottleneck for finding input bottlenecks, so compare epoch times and GPU utilisation before and after the change.
- State whether you train on a GPU or only on a CPU.
- Share the current DataLoader arguments and transform list.
- Ask for one change at a time so you can measure each effect.
What people use it for
- Setting up a DataLoader for an image classification training run in PyTorch.
- Reviewing an existing preprocessing script that leaves the GPU idle between batches.
- Deciding whether to cache preprocessing results or move a large dataset to memory-mapped files or WebDataset.
- Switching image transforms to torchvision.transforms.v2 during a code cleanup.
Frequently asked questions
Is the data-loading skill free to use?
The source lists the licence as MIT. Check the repository's licence file for the full terms before you reuse or redistribute it.
What do I need to use the data-loading skill?
You need PyTorch, because the advice refers to DataLoader and torchvision.transforms.v2. The skill runs no scripts and uses no network, so it is only guidance for the agent.
How do I install the data-loading skill?
Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use or a project's .claude/skills/ folder. You can also upload the ZIP under Settings → Capabilities → Skills in the Claude apps.
Will this skill make my training faster?
It gives rules of thumb such as setting num_workers, pinning memory and using persistent workers. The source includes no benchmarks, so you need to measure the effect on your own hardware.
Does this work for non-PyTorch data pipelines?
No. The advice is specific to DataLoader and torchvision, and it may mislead in other frameworks. Very small datasets or CPU-only jobs may also gain little.
What is an alternative if this skill does not fit my project?
The source does not name alternatives. You can ask the agent directly for DataLoader tuning help, or read the PyTorch Data Loading Tutorial that the skill cites as its reference.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you train PyTorch models and want your agent to apply a clear checklist when setting up or reviewing a DataLoader. Pick something else if your work uses another framework, runs on a CPU only with small data, or needs measured performance results, since this file gives rules of thumb only.
Download the skill