Skip to content
Get packs

Guide · Open source

How to use ds4: build, download a model and run it locally

Updated · written with AI from the project's own documentation and checked against our catalogue data

ds4 (DwarfStar, repo antirez/ds4) is a small native C program that runs specific large language models on your own computer instead of a cloud service. It is built mainly around DeepSeek V4 Flash and a few related models, and targets Apple Metal, NVIDIA CUDA and AMD ROCm hardware. This guide shows you how to clone and build it, download a model, and use its command-line chat, local server and coding agent.

DwarfStarantirez/ds4 · ★ 24.2k · MIT

Quick start

  1. 1Check your hardware. Confirm you have enough memory for the model you want, using the memory guidance in the platform guide and README.
  2. 2Clone the repository. Run git clone https://github.com/antirez/ds4.git and then cd ds4.
  3. 3Read your platform guide. Open the guide in docs/ that matches your hardware, such as docs/METAL.md for Apple Silicon.
  4. 4Build. Run the build command for your platform, for example make on Apple Silicon.
  5. 5Download a model. Run ./download_model.sh ds4f-q2 to fetch DeepSeek V4 Flash Q2 into the gguf/ folder.
  6. 6Run it. Start ./ds4 for chat, ./ds4-server for the local HTTP server, or ./ds4-agent for the coding agent.

Check whether your hardware is suitable

Before installing, confirm that your machine can hold the model you want. The README says Metal on Macs needs 64 GB for Qwen Flash next and 96 GB or more for DeepSeek and GLM. Smaller machines can use SSD streaming, though the README notes that 96 GB and 128 GB sizes are what most models need.

The project lists three main hardware targets: Metal on Apple Silicon, NVIDIA CUDA (including the DGX Spark and Ada Lovelace cards such as the L40S), and ROCm on Strix Halo systems such as the Framework Desktop.

  • Apple Silicon Mac: 64 GB or more, 96 GB or more for DeepSeek and GLM
  • NVIDIA CUDA: DGX Spark or CUDA cards, including Ada Lovelace/L40S
  • AMD ROCm: Strix Halo systems such as the Framework Desktop

Clone and build ds4

Clone the repository and enter the project folder. Then choose the build command that matches your hardware. Each platform has its own guide covering prerequisites, memory sizing and hardware-specific setup, so read that guide before building.

  • Metal on Apple Silicon: docs/METAL.md, build with make
  • DGX Spark: docs/DGX_SPARK.md, build with make cuda-spark
  • Strix Halo / Framework Desktop: docs/STRIX_HALO.md, build with make strix-halo
  • One or more CUDA cards, including Ada/L40S: docs/CUDA_MULTI_GPU.md, build with make cuda-generic
git clone https://github.com/antirez/ds4.git cd ds4 make

Download a model

For a first run on a 96 or 128 GB machine, the README suggests downloading DeepSeek V4 Flash Q2. Downloads are saved in the gguf/ folder. If a download is interrupted, run the same command again to resume it.

Leave memory free for the context and runtime buffers as well as the model itself. For other models, see docs/MODELS.md. On a smaller Mac, the README points to docs/SSD_STREAMING.md.

  • Downloads go to gguf/
  • Re-run the command to resume an interrupted download
./download_model.sh ds4f-q2

Run the command-line chat

Once built and with a model downloaded, run ./ds4 for an interactive chat. It keeps a multi-turn conversation. Pass a prompt with -p to get a single answer instead.

By default ds4 uses ds4flash.gguf, a link updated by main-model downloads. Pass -m FILE to pick a model explicitly. Commands normally run from the repository root; use --chdir /path/to/ds4 when launching from elsewhere. Run each binary with --help to see all options.

  • /help shows the in-chat commands
  • /read FILE loads a file into the conversation
  • /ctx N sets the context size
  • /quit exits; Ctrl+C interrupts generation and returns to the prompt
./ds4 ./ds4 -p "Explain Redis streams in one paragraph."

Run the local HTTP server

ds4-server exposes the model over HTTP. By default it listens at http://127.0.0.1:8000. The README points to docs/SERVER.md for API access and multiple sessions. You can set the context size with --ctx.

For tools such as Pi, OpenCode, Codex CLI or Claude Code, use ds4-server rather than the agent, and follow docs/CLIENTS.md.

./ds4-server --ctx 32768

Use the native coding agent

ds4-agent runs inference directly, without a separate HTTP server. It keeps token history and model state together, shows prefill progress and uses the model's native tool format.

Sessions are saved in ~/.ds4/kvcache. Use /save, /list, /switch <sha> and /del <sha> to manage them. /strip <sha> keeps the text and title but removes the large KV payload. Saved conversations and traces may contain private information, so treat that folder with care.

  • /hints on and /hints off control brief explanations of programming choices
  • Sessions containing images cannot yet be saved
  • Stripped sessions and network TP restores require prefill
./ds4-agent

What people use it for

  • Chat with a large model on a high-memory Mac without sending prompts to a cloud service
  • Serve a local model to several users or sessions on CUDA hardware, as the project describes for multi-GPU setups
  • Connect a coding client such as Pi, OpenCode or Codex CLI to the local server
  • Ask the native coding agent to adapt the code to your own hardware, which the README encourages

Frequently asked questions

Is ds4 free to use?

The project is released under the MIT licence, according to the directory analysis of the repository. The license covers the code, but you still need hardware with enough memory to run the models.

What hardware do I need to run ds4?

The README recommends a Mac with 64 GB for Qwen Flash next and 96 GB or more for DeepSeek and GLM on Metal. Smaller machines can use SSD streaming, and CUDA or Strix Halo systems have their own platform guides.

How do I install ds4?

Clone the repository with git clone https://github.com/antirez/ds4.git, enter the ds4 folder, and run the build command for your platform. On Apple Silicon that is make.

Is ds4 a general GGUF runner for any model?

No. The README says it is deliberately narrow and uses only the GGUF files the project produces. It is not meant to run arbitrary GGUF models.

Is ds4 production ready?

The README calls the software beta quality and says it changes very fast, so instabilities and regressions are possible. Teams that need production stability should choose something else.

How does ds4 compare to llama.cpp, Ollama or LM Studio?

The README acknowledges llama.cpp and GGML as the foundation it builds on. ds4 focuses on a few specific models, while tools like llama.cpp, Ollama, vLLM and LM Studio are broader options that run many models.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

ds4 suits developers with a 64 GB or larger Mac, NVIDIA CUDA owners, and tinkerers who are comfortable with AI coding agents and building from source. It is a poor fit if your machine is small, you want a general GGUF runner, you object to AI-assisted code, or you need production stability. For those cases, look at general runners such as llama.cpp, Ollama or LM Studio.

Install options & downloads