Skip to content
Get packs

Dataset Curation

Format, template, pack, and validate training datasets, and write the dataset card, before fine-tuning

Source Download skill
Repository
wshobson/agents
Stars
42,095
Licence
MIT
Files
3 files

What it does

This skill guides an AI agent through preparing data for fine-tuning a language model: choosing the right row format for the training method, applying the model's chat template correctly, packing short examples efficiently, and mixing in synthetic (model-generated) data without degrading quality. It also requires a completed dataset card documenting provenance, counts, dedup method, template and packing settings before training starts.

Great for

  • Engineers preparing instruction or chat data for supervised fine-tuning or preference training
  • Teams that need a documented, auditable dataset card before launching a training run
  • People generating synthetic training data who want guardrails against model collapse

Skip it if

  • Users who have not yet chosen a fine-tuning method; the skill assumes routing from a separate finetuning-method-selection skill and will not help choose the method itself
  • Quick prompt engineering or one-off chat use, where fine-tuning is unnecessary and the setup adds effort without benefit
  • Anyone relying on the skill's numeric guidance (row floors, accept rates, efficiency multipliers) without checking it against their own model and data, since these are stated as rules of thumb

Try asking

  • “Convert my customer support transcripts into ChatML messages format for SFT and check the chat template is applied correctly”
  • “Set up sequence packing for my 3,000-example instruction dataset and show me how to inspect packed sequences before training”
  • “Generate synthetic math word problems with rejection sampling and build a replay mix that keeps at least 25% real data”
  • “Write the dataset card for my fine-tuning run, including provenance, split counts, dedup method and template string”
Step-by-step guide: how to use Dataset CurationInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 3 original files, the MIT licence and an attribution note.

    dataset-curation.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip dataset-curation.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip dataset-curation.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Decode and manually inspect 5-10 packed sequences before a full run, as the skill mandates, because packing bugs often do not show up in the loss curve.
  • Name exactly which rows count as real data in the dataset card, including general-domain replay rows, so the 25% real-data check is meaningful.
  • Use the same chat template string for training, evaluation and inference, and record it in the card.

What's inside

  • references/formats-and-templates.md
  • references/synthetic-data.md
  • SKILL.md

Redistributed unmodified from wshobson/agents under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.