Skip to content
Get packs

Hugging Face Local Models

Find GGUF models on the Hugging Face Hub and run them locally with llama.cpp

Source Download skill
Repository
huggingface/skills
Stars
11,253
Licence
Apache-2.0
Files
4 files

What it does

Once installed, the agent can search the Hugging Face Hub for models packaged in GGUF format, which are built to run on everyday hardware, and pick a suitable compression level (quant) for your memory or graphics card. It then gives you the commands to launch the model with llama.cpp, either as a chat tool or as a local server that other apps can call in the same way as OpenAI's API. It can also convert a model to GGUF when no packaged version exists.

Great for

  • Developers who want to run open models privately on their own laptop, desktop or GPU
  • People setting up a local OpenAI-compatible endpoint for an app they are building
  • Users with limited RAM or VRAM who need help choosing the right model size and quantization

Skip it if

  • Users who need the most capable cloud models, since local models on consumer hardware are usually smaller and less accurate (the skill does not change that)
  • Anyone who expects a one-click setup, because it assumes llama.cpp is installed and comfort with the terminal
  • Gated or private models without a Hugging Face account and access approval, since authentication must be set up manually with hf auth login

Try asking

  • “Find a good GGUF coding model that fits in 16 GB of RAM and give me the llama-server command”
  • “Show me the exact .gguf filenames available for unsloth/Qwen3.6-35B-A3B-GGUF”
  • “Convert this Transformers model repo to GGUF and quantize it to Q4_K_M”
  • “Start a local OpenAI-compatible server on port 8080 and test it with a curl request”
Step-by-step guide: how to use Hugging Face Local ModelsInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 4 original files, the Apache-2.0 licence and an attribution note.

    huggingface-local-models.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip huggingface-local-models.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip huggingface-local-models.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Let the agent confirm exact .gguf filenames via the Hub tree API before launching, so you do not download the wrong quant
  • State your RAM, VRAM and GPU type (Metal, CUDA, ROCm or CPU) up front so the quant recommendation fits your hardware
  • Review any build or convert commands before running them, since they download and execute code and large model files

What's inside

  • references/hardware.md
  • references/hub-discovery.md
  • references/quantization.md
  • SKILL.md

Redistributed unmodified from huggingface/skills under the Apache-2.0 licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.