Guide · Agent skill
How to use the Quantized Model Export agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The quantized-export skill guides an AI agent through exporting a fine-tuned language model that has already passed a promotion gate into a format suited to its target hardware, such as FP8, AWQ INT4 or GGUF. It is written for ML engineers and teams who deploy fine-tuned models and want a consistent export step. After this guide you will know what the skill does, how to install it, how to ask for it in plain language, and what it needs before it can run.
Quantized Model Exportwshobson/agents · MIT Download the skill Quick start
- 1Download and unzip. Download the quantized-export ZIP from this page and unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder.
- 2Prepare your inputs. Have a promoted checkpoint or LoRA adapter, your target runtime, and an eval/goldens.jsonl file ready.
- 3Describe the target. Ask the agent for an export, naming your GPU or device, serving stack, and whether long-context, code, or math workloads are in scope.
- 4Follow the format pick. Let the agent choose FP8, AWQ INT4, or GGUF Q4_K_M with imatrix based on the target, and keep INT4 off long-context, code, and math.
- 5Export using the reference commands. Use the runnable sequences in references/export-commands.md, filling in CHECKPOINT_DIR, MERGED_DIR, GGUF_DIR, and BASE_MODEL.
- 6Run the smoke test. Load the export in its target runtime and run python smoke_test.py against your goldens and pre-export outputs.
What is an agent skill, and what does quantized-export make the agent do?
An agent skill is a folder with a SKILL.md file that tells an AI agent how to handle a specific kind of task. When a request matches the skill's description, the agent reads the instructions and follows them.
The quantized-export skill covers the step after a checkpoint has been promoted. It helps the agent choose a format by hardware and deployment shape, warns about workloads that break under INT4, and requires a smoke test that compares outputs before and after export.
- Picks a format such as FP8, AWQ INT4, or GGUF with Q4_K_M and imatrix, based on the target GPU or edge device.
- Flags long-context, code, and math workloads as unsafe for INT4 and keeps them on FP8 or W8A8.
- Chooses between a merged export and a LoRA-only export.
- Runs a smoke test on 3 to 5 golden prompts and compares them with pre-export outputs.
- Points to references/export-commands.md for runnable command sequences.
Install the quantized-export skill
Download the skill ZIP from this page and unzip it. For a personal setup, place the folder in ~/.claude/skills/. For a single project, place it in that project's .claude/skills/ folder.
You can also upload the skill in the Claude apps. Go to Settings, then Capabilities, then Skills, and upload the ZIP there.
- Personal install: ~/.claude/skills/quantized-export/
- Project install: .claude/skills/quantized-export/ inside your project
- Claude apps: Settings → Capabilities → Skills, then upload the ZIP
Check the requirements before you start
The skill assumes you already have a trained or fine-tuned checkpoint. It is not a training tool, and it does not create a model from scratch. The skill also expects the output of an earlier promotion step and an eval file.
The companion checkpoint-promotion and eval-harness-first skills are referenced by this skill but are not included in the download.
- Python, because the smoke test script is run with python.
- A promoted checkpoint or LoRA adapter.
- A target runtime such as vLLM or llama.cpp.
- An eval/goldens.jsonl file, typically maintained by the eval-harness-first skill.
- The checkpoint-promotion and eval-harness-first skills, which are referenced but not included.
How to trigger the skill with natural requests
You do not need a special command. Describe the export task in plain language that matches the skill's purpose: after a checkpoint passes promotion, when choosing a quantization format for a target device, or when an exported model fails its smoke test.
Name your hardware and workload when you can. The skill's format choice depends on them.
- "My checkpoint passed promotion. Export it for an H100 serving generic chat with vLLM."
- "Which quantization format should I use for a GGUF build on a laptop running llama.cpp?"
- "My exported model gives garbled output. Run the smoke test and help me find the cause."
- "Should I merge the LoRA adapter or keep it separate for multi-adapter serving?"
Run the smoke test on every export
A file that loads is not proof that an export works, because malformed exports still load. The skill makes the smoke test mandatory. It loads the artifact in its real target runtime, runs golden prompts, and compares the results with the pre-export generations using greedy decoding and a fixed seed.
For lossless exports, any byte difference is a bug. For quantized exports, byte match is expected to fail, so the gate becomes task-grader agreement instead.
- Template mismatch looks like garbled or run-on output.
- A wrong quantization on lm_head looks fluent but semantically off.
python smoke_test.py "$EXPORT_PATH" \
eval/goldens.jsonl pre-export-outputs.jsonl
# non-zero exit on any pre/post mismatchSafety notes: scripts and network access
This skill runs scripts, so read them before you run an export on a machine with sensitive data. It does not use the network, according to its listing. Export commands read a promoted checkpoint and write an artifact to the paths you give them.
Fill in the placeholders CHECKPOINT_DIR, MERGED_DIR, GGUF_DIR, and BASE_MODEL with your own paths before running any command from the reference file.
- The skill runs scripts, including smoke_test.py.
- The listing states it uses no network access.
- Use the exact same base model revision for LoRA-only serving, since a mismatched base changes outputs silently.
Tips for better exports
Pick the format from the hardware and workload, not habit. FP8 is the default on Hopper-class GPUs and newer. Do not choose NVFP4 on GB10, because the skill says it runs about 32% slower than FP8 there.
Re-run the smoke test after any quant-method or runtime version bump, not only after the first export.
- Measure with the real task evals from eval-harness-first, not MMLU alone.
- If an INT4 export regresses on long-context, code, or math, switch format instead of re-tuning the recipe.
- Avoid GPTQ for new deployments, since the skill says AWQ has better accuracy retention.
What people use it for
- Exporting a promoted checkpoint to FP8 for a datacenter GPU serving generic chat.
- Building a GGUF Q4_K_M file with imatrix for local llama.cpp use on a laptop or edge device.
- Deciding whether to ship a merged artifact or a LoRA-only adapter for serving.
- Diagnosing an exported model that gives garbled or off-template output by running the smoke test.
Frequently asked questions
Is the quantized-export skill free to use?
The skill is listed under the MIT licence. The source does not mention pricing, so check the licence text for the terms that apply.
What do I need before I can use quantized-export?
You need Python, a promoted checkpoint or LoRA adapter, a target runtime such as vLLM or llama.cpp, and an eval/goldens.jsonl file. The checkpoint-promotion and eval-harness-first skills are referenced but not included in the download.
Can I use this skill without a fine-tuned model?
No. The skill assumes a promoted checkpoint as input, so it is not useful if you have no trained or fine-tuned model.
Which format should I pick for an older GPU?
The skill recommends AWQ INT4 for older GPU generations that predate FP8 hardware support. It also says GPTQ is superseded for new deployments.
Does the skill work for NVFP4 on GB10?
The skill says not to use NVFP4 on GB10. It reports that NVFP4 there runs about 32% slower than FP8, so it recommends FP8 instead.
Does a successful file export mean my model works?
No. The skill states that malformed exports still load, so you should run the smoke test against your golden prompts before deploying.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use quantized-export if you are an ML engineer or team with a promoted fine-tuned checkpoint and need a repeatable export and smoke-test step for datacenter, older GPU, or llama.cpp targets. Pick something else if you do not have a trained model, or if you need a general-purpose training or serving tool, since this skill only covers the export stage.
Download the skill