Skip to content
Get packs

Guide · Agent skill

How to use the Model Pruning for LLMs agent skill

Updated · written with AI from the project's own documentation and checked against our catalogue data

The model-pruning skill is a set of instructions and code templates that teaches an AI agent how to shrink a large language model by zeroing out its less important weights. It is aimed at ML engineers and researchers who work with open-weight models and want to reduce memory use or speed up inference before deployment. After reading this guide you will know what the skill does, how to install it, how to ask for it in plain language, and what you need in place before using it.

Model Pruning for LLMsOrchestra-Research/AI-Research-SKILLs · MIT

Quick start

  1. 1Download and unzip the skill. Download the ZIP from our page and unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder.
  2. 2Prepare your environment. Install Python, PyTorch, transformers and accelerate, and make sure Git is available.
  3. 3Get access to model weights. Confirm you can load a model such as meta-llama/Llama-2-7b-hf, including any Hugging Face account step or licence acceptance.
  4. 4Describe your goal. Ask the agent to prune your model, naming the technique and target sparsity such as Wanda at 50%.
  5. 5Review the generated code. Read the scripts and clone commands before running anything on your machine.
  6. 6Run and save the pruned model. Run the approved code on a GPU, then save the pruned model to a local folder.
  7. 7Evaluate accuracy. Measure accuracy on your own tasks, because the skill does not check accuracy loss for you.

What the model-pruning skill makes the agent do

Pruning removes weights from a neural network by setting them to zero. The skill covers one-shot methods such as Wanda and SparseGPT, which compress a model without retraining it and only need a small calibration dataset to collect activation statistics.

The skill also explains the trade-offs between unstructured pruning, structured pruning and N:M sparsity. Unstructured pruning gives the best accuracy but no hardware speedup, structured pruning is hardware friendly but loses more accuracy, and N:M sparsity such as 2:4 aims for both speed and quality on NVIDIA sparse tensor cores.

  • Wanda: importance is weight magnitude multiplied by input activation.
  • SparseGPT: second-order pruning that uses Hessian information.
  • Magnitude pruning: the simple baseline that uses weight size alone.
  • N:M sparsity: keep N weights in every group of M, for example 2 of every 4.

How to install the model-pruning skill

Download the ZIP for this skill from our page. Then either unzip it into ~/.claude/skills/ for your user account, or unzip it into a project's .claude/skills/ folder to make it available only in that project.

You can also upload the skill in the Claude apps under Settings → Capabilities → Skills.

  • Personal install: unzip into ~/.claude/skills/
  • Project install: unzip into .claude/skills/ inside your project
  • Claude apps: Settings → Capabilities → Skills, then upload

How to trigger the skill with natural requests

You do not need a special command. Describe the goal in plain words that match the skill's purpose, such as compressing a model without retraining or reaching 50% sparsity. The skill's description covers compressing models, reaching 50% sparsity with minimal accuracy loss, and faster inference on hardware accelerators.

Name the technique if you know it, for example Wanda, SparseGPT or 2:4 sparsity, so the agent picks the right section.

  • Compress Llama-2-7b without retraining using Wanda at 50% sparsity.
  • Show me how to apply 2:4 N:M sparsity for NVIDIA sparse tensor cores.
  • Compare magnitude, Wanda and SparseGPT pruning for my model.

Example: one-shot Wanda pruning

The skill includes a Wanda template. It loads a model with Hugging Face transformers, runs a few calibration texts through it with forward hooks to record activation statistics, then zeroes weights whose |weight| × activation falls below a quantile threshold at the chosen sparsity. The template saves the result to a local folder.

The snippet is illustrative. Treat it as a starting point and check the output yourself before relying on it.

pruned_model = wanda_prune(model, calib_data, sparsity=0.5) pruned_model.save_pretrained("./llama-2-7b-wanda-50")

What you need before you start

The workflow is code-heavy. You need Python, PyTorch, the Hugging Face transformers and accelerate libraries, and Git to clone the wanda and sparsegpt repositories. A CUDA-capable GPU is recommended for large models.

You also need access to model weights such as meta-llama/Llama-2-7b-hf. That model may require a Hugging Face account and acceptance of its licence.

  • Python, PyTorch, transformers, accelerate
  • Git, to clone the wanda and sparsegpt repositories
  • CUDA-capable GPU recommended for large models
  • Access to model weights, possibly needing a Hugging Face account and licence acceptance
pip install torch transformers accelerate

Safety notes: scripts and network access

This skill runs scripts and uses the network. Its instructions clone code from GitHub and load model weights from Hugging Face, so review the repositories and scripts before running them on your machine.

The skill does not measure accuracy loss for you. After pruning, evaluate the model yourself, since the snippets are illustrative and not validated for production use.

  • Review cloned repositories and scripts before executing them.
  • Re-evaluate pruned models for accuracy loss on your own tasks.
  • Do not treat the templates as production-ready code.

Tips for better results

Pick the pattern to match your hardware first. If you plan to use NVIDIA 2:4 sparse tensor cores, ask for N:M sparsity; if you need flexibility and do not need a speedup, unstructured pruning keeps more accuracy.

Keep the calibration set small but representative of your real inputs, and compare at least two criteria so you can see the effect of the method on your model.

  • Match sparsity pattern to your hardware.
  • Use a representative calibration set.
  • Compare magnitude, Wanda and SparseGPT on the same model.
  • Save each pruned variant in its own folder so results are easy to compare.

What people use it for

  • Shrinking an open-weight 7B model so it fits in less GPU memory before deployment.
  • Testing 2:4 sparsity on an NVIDIA GPU to see whether inference gets faster.
  • Comparing magnitude, Wanda and SparseGPT pruning on the same model for a research write-up.
  • Building a one-shot compression step into a deployment pipeline before evaluation.

Frequently asked questions

How do I install the model-pruning skill?

Download the ZIP from our page and unzip it into ~/.claude/skills/ for your user account, or into a project's .claude/skills/ folder. You can also upload it in the Claude apps under Settings → Capabilities → Skills.

What do I need to run the model-pruning skill?

You need Python, PyTorch, Hugging Face transformers and accelerate, and Git to clone the wanda and sparsegpt repositories. A CUDA-capable GPU is recommended for large models, and you need access to model weights such as meta-llama/Llama-2-7b-hf.

Is the model-pruning skill free?

The skill is published under the MIT licence, as stated on the source page. The model weights you use may have their own licence terms, such as Llama-2 requiring acceptance on Hugging Face.

Does pruning need retraining?

The one-shot methods covered here, Wanda and SparseGPT, compress a model without retraining. They use a small calibration dataset to gather statistics before pruning.

What is the difference between Wanda and SparseGPT?

Wanda ranks weights by magnitude multiplied by input activation, while SparseGPT uses second-order information from the Hessian. SparseGPT is described as more accurate but more computationally expensive.

Why is my pruned model not faster?

Unstructured sparsity gives no hardware speedup because the zero pattern is irregular. For a speedup on NVIDIA GPUs, the skill points to N:M sparsity such as 2:4 on sparse tensor cores.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Choose this skill if you are an ML engineer or researcher who wants to compress an open-weight LLM with one-shot methods and you are comfortable with Python, PyTorch and a GPU. Skip it if you only want to run or chat with an existing model, or if you need a validated production pipeline, since the snippets are illustrative and accuracy checks are up to you.

Download the skill