Axolotl LLM Fine-Tuning Guide
Guides an agent through fine-tuning LLMs with Axolotl: YAML configs, LoRA/QLoRA, DPO and multi-GPU setups.
- Repository
- Orchestra-Research/AI-Research-SKILLs
- Stars
- 14,195
- Licence
- MIT
- Files
- 3 files
What it does
Once installed, the agent knows the Axolotl open-source tool for training and fine-tuning large language models. Axolotl is configured through YAML files rather than custom code, and this skill gives the agent patterns for common choices: parameter-efficient methods like LoRA and QLoRA, preference-tuning methods such as DPO, KTO, ORPO and GRPO, distributed training with FSDP or DeepSpeed, and dataset formats. The skill is mostly a set of reference notes and short code snippets pulled from the official documentation, so the agent can answer questions and draft configs more accurately. It does not itself run any training.
Great for
- ML engineers who are setting up or debugging an Axolotl fine-tuning job
- Developers who want an agent to draft a YAML config for a specific model and dataset format
- Teams planning multi-GPU training who need guidance on FSDP, DeepSpeed or context parallelism
Skip it if
- Non-technical users: fine-tuning needs GPUs, dataset preparation and familiarity with ML tooling, and the skill cannot provide those.
- Anyone expecting verified, up-to-date answers: the SKILL.md says it was auto-generated from documentation and some sections are incomplete (for example, a context_parallel_size pattern is only a bare name with no example).
- Production decisions based solely on this skill: check the official Axolotl docs and your own test results, since the content is a scraped summary.
Try asking
- “Write an Axolotl YAML config to LoRA fine-tune a Llama model on my instruction dataset.”
- “Explain how context_parallel_size changes the global batch size on my 8-GPU machine.”
- “Show me the dataset format I need for DPO training in Axolotl.”
- “Help me enable FSDP version 2 in my Axolotl config and check the options.”
Install in 3 steps
- 1
Download the skill
The ZIP contains the 3 original files, the MIT licence and an attribution note.
axolotl.zip - 2
Add it to your agent
Claude Code (all projects)
mkdir -p ~/.claude/skills && unzip axolotl.zip -d ~/.claude/skills/Claude Code / Agent SDK (this project only)
mkdir -p .claude/skills && unzip axolotl.zip -d .claude/skills/Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.
- 3
Ask for the task
No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.
Get better results
- Ask the agent to quote the relevant reference file section so you can verify settings against the official Axolotl docs.
- Give the agent your model name, GPU count and dataset format up front so it can tailor the YAML rather than give generic examples.
- Treat generated configs as drafts: run a small test job before committing GPU hours to a full training run.
What's inside
- references/dataset-formats.md
- references/index.md
- SKILL.md
Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.