Skip to content
Get packs

Model Pruning for LLMs

Shrink and speed up language models by pruning weights, without retraining, using Wanda, SparseGPT and N:M sparsity

Source Download skill
Repository
Orchestra-Research/AI-Research-SKILLs
Stars
14,195
Licence
MIT
Files
2 files

What it does

When installed, this skill gives the agent guidance and code templates for making a large language model smaller and faster by removing (zeroing out) less important weights, a process called pruning. It covers one-shot methods like Wanda and SparseGPT that need only a small calibration dataset rather than full retraining, and explains trade-offs between unstructured, structured and N:M sparsity patterns that suit specific hardware accelerators.

Great for

  • ML engineers who want to compress an open-weight LLM to reduce memory use before deployment
  • Developers targeting NVIDIA GPUs with 2:4 sparse tensor cores who want inference speedups
  • Researchers comparing pruning criteria such as magnitude, Wanda and SparseGPT

Skip it if

  • Users who just want to run or chat with an existing model, since no pruning is needed (the skill adds nothing there)
  • Production use without validation: the snippets are illustrative, and pruned models must be re-evaluated for accuracy loss, which the skill does not measure for you
  • Non-technical users, because the workflow requires Python, PyTorch and access to large model weights and a GPU

Try asking

  • “Help me prune Llama-2-7b to 50% sparsity with Wanda and save the result”
  • “Explain the difference between unstructured and 2:4 N:M sparsity for my NVIDIA GPU”
  • “Write a script that applies SparseGPT to my Hugging Face model using calibration data”
  • “Which pruning method should I use if I need no retraining and minimal accuracy loss?”
Step-by-step guide: how to use Model Pruning for LLMsInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 2 original files, the MIT licence and an attribution note.

    model-pruning.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip model-pruning.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip model-pruning.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Review any generated pruning script before running it, especially code that overwrites model weights, and keep an unpruned copy of the model.
  • Measure perplexity or task accuracy on a held-out set after pruning, since the claimed accuracy retention (<1%) is not verified by this skill.
  • Check the referenced GitHub repositories and their install instructions directly, as the skill's snippets are simplified and may not match the official implementations.

What's inside

  • references/wanda.md
  • SKILL.md

Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.