How to use the Fine-Tuning Method Selection agent skill

What the skill makes the agent do

Before any training starts, the skill checks whether retrieval (RAG) or prompt engineering would solve the problem more cheaply. Facts that change often, such as prices, docs or news, are routed to RAG, because a fine-tuned model bakes in a snapshot that goes stale quickly.

If training is justified, the skill follows a decision tree. The data shape in hand decides the method: input/output demonstrations point to SFT, preference pairs point to DPO/ORPO, unpaired thumbs-up/down points to KTO, and tasks with a verifiable pass/fail signal point to GRPO+RLVR. It also gives rough sizing guidance for model size and memory.

• Routes new facts to RAG or continued pretraining (CPT), and new behavior to prompt engineering or a training method.
• Sizes memory with a worksheet: weights, optimizer states, gradients and activations.
• Requires an evaluation harness to exist before any method is chosen.

Install the skill in Claude

You can install the skill in two ways. Download the ZIP from the skill's page on this site and unzip it into your user skills folder, ~/.claude/skills/, so it is available in every project. Or unzip it into a single project's .claude/skills/ folder, so it applies only there.

If you use the Claude apps instead of a project folder, upload the ZIP under Settings → Capabilities → Skills. The skill's folder contains SKILL.md plus two reference files, references/memory-math.md and references/model-catalog.md, which the skill reads when it needs sizing or model-class detail. Keep that folder structure intact when you unzip.

• User-wide: ~/.claude/skills/
• Single project: .claude/skills/ inside the project
• Claude apps: Settings → Capabilities → Skills, then upload the ZIP

How to trigger the skill with natural requests

You do not need a special command. The skill is meant to load when your request matches its description: starting any fine-tuning effort, unsure whether RAG or prompting would suffice, or choosing between preference-optimization and reinforcement methods. Describe your situation in plain language and include the kind of data you have, because the data shape drives the answer.

Requests that work well name the problem and the data. For example, say what the users need the model to do, and whether you have transcripts, pairs of good and bad answers, or an automatic way to grade correctness.

• "Should we fine-tune our assistant to follow our support macros, or is prompting enough?"
• "We have reviewer thumbs-up and thumbs-down on responses, unpaired. Which method fits?"
• "Our model solves some math problems and we can grade answers automatically. Is GRPO worth trying?"
• "We want the model to know this week's pricing page. Do we need to train anything?"

Read the routing answer

The skill's answer follows the decision tree from top to bottom: first decide whether the gap is new facts or new behavior, then follow the branch that matches your data. Demonstrations lead to SFT, preference pairs to DPO (SimPO if length bias is a concern, ORPO if memory is tight), unpaired thumbs-up/down to KTO, and verifiable success to RLVR with GRPO.

The skill also warns about common mistakes. Choosing a DPO variant is a small lever compared with data quality and model scale, and RL should not start on a model that fails every attempt. If the model cannot yet do the task, the skill routes you to SFT first.

• Volatile facts: RAG, no training run.
• Stable behavior with demonstrations: SFT.
• Paired preferences: DPO. Unpaired thumbs-up/down: KTO.
• Verifiable pass/fail signal: GRPO+RLVR, but only once the model succeeds at least sometimes.

Estimate memory before committing to a run

Total memory is roughly parameters times bytes per parameter, plus optimizer state, gradients and activations. The skill's memory reference gives bytes per parameter by dtype: 4 for fp32, 2 for bf16 or fp16, 1 for int8, and 0.5 for int4 (QLoRA NF4). The reference describes this as planning math, so leave headroom rather than sizing to the byte.

The skill names model size classes only, such as an 8B-class LoRA run, and does not name specific base models. The current model choices for each size class live in a separate model catalog file, which is dated and should be checked before you rely on any row.

• Weights = params × bytes per parameter.
• Full fine-tuning is dominated by the weights term; LoRA and QLoRA keep the adapter term near zero.
• On DGX Spark, the skill defers to the dgx-spark-ops plugin's spark-memory-thermal-ops skill, if installed.

Hand off to the method skill

This skill decides and sizes; it does not write training code. Once a method is picked, the skill points you to the skill that executes it: lora-qlora-recipes for SFT via LoRA or QLoRA, preference-optimization for DPO, ORPO and KTO, and grpo-rlvr-training for GRPO with verifiable rewards. Set up the eval harness first, as the skill requires, using the eval-harness-first skill it references.

• SFT: lora-qlora-recipes
• DPO, ORPO, KTO: preference-optimization
• GRPO with verifiable rewards: grpo-rlvr-training