Guide · Open source
How to use nanochat: install, train GPT-2 and chat
Updated · written with AI from the project's own documentation and checked against our catalogue data
nanochat is an open-source codebase for training a large language model end to end, covering tokenization, pretraining, finetuning, evaluation and inference. This guide explains how to install it with uv, run the reference GPT-2 training script on a GPU node, and chat with the resulting model from the command line. It also covers the reduced CPU and Apple Silicon path and what to expect from it.
nanochatkarpathy/nanochat · ★ 59.6k · MIT Install options & downloads Quick start
- 1Clone the repository. Get the nanochat source code from GitHub and open the project folder.
- 2Install dependencies. Run uv sync with --extra gpu for CUDA GPUs, or --extra cpu for CPU-only or MPS machines.
- 3Activate the environment. Run source .venv/bin/activate so the installed packages are available.
- 4Run the GPT-2 speedrun. On an 8XH100 node, run bash runs/speedrun.sh. The README estimates about 1.5 hours and about $48 of cloud time.
- 5Chat with the model. Start python -m scripts.chat_cli and ask it questions in the terminal.
- 6Tune for your VRAM. If you have less than 80GB per GPU, lower --device-batch-size from 32 until the run fits.
What you need before you install nanochat
nanochat uses uv, a Python dependency manager, for its setup. The README expects Python, uv, and either a CUDA-capable GPU or a CPU/MPS machine for a reduced run.
The full GPT-2 reproduction is designed for a single 8XH100 GPU node. The README notes that GPUs with less than 80GB of memory will need hyperparameter tuning to avoid running out of VRAM.
- uv (Python dependency manager)
- Python
- A CUDA-capable GPU for the full run, or CPU/MPS for a reduced run
- For the GPT-2 reproduction: an 8XH100 GPU node (the README says an Ampere 8XA100 node also works, but slower)
Install nanochat with uv
Clone the repository, then install dependencies with the extra that matches your hardware. Use the GPU extra for CUDA GPUs such as A100 or H100, and the CPU extra for CPU-only or Apple Silicon (MPS) machines. Then activate the virtual environment.
If you plan to develop the code, the README also lists a dev group that adds pytest, matplotlib, ipykernel and transformers.
uv sync --extra gpu # Use for CUDA (A100/H100/etc.)
uv sync --extra cpu # (or) Use for CPU-only / MPS
source .venv/bin/activateTrain a GPT-2 class model with runs/speedrun.sh
The entire pipeline for training a GPT-2 capability model is in one file, runs/speedrun.sh. The README says it is designed for an 8XH100 GPU node and takes about 1.5 hours to run, so you may want to run it inside a screen session. The README estimates the cost at about $48 of cloud GPU time, and closer to $15 on a spot instance.
Launch the script from the repository root once your node is ready and your environment is set up.
- Boot an 8XH100 GPU node from a cloud provider. The README mentions Lambda as one option.
- Run the script inside a screen session if your connection may drop.
- Expect roughly 1.5 hours of runtime.
bash runs/speedrun.shChat with your model from the command line
When training finishes, make sure your uv virtual environment is active, then start the command-line chat. The README suggests asking it to write stories or poems, or asking why the sky is blue, to see what it does.
Set expectations: the speedrun model is a roughly 4e19 FLOPs capability model, and the README says talking to it is a bit like talking to a kindergartener.
source .venv/bin/activate
python -m scripts.chat_cliAdjust settings for smaller GPUs
If your GPU has less than 80GB of VRAM, reduce the --device-batch-size value. The README says the default is 32, and you can try 16, 8, 4, 2 or 1. Below that, the README says you need more experience to tune the run.
The code also runs on a single GPU if you omit torchrun. The README says results will be nearly identical, with gradient accumulation used automatically, but training takes about eight times longer.
- Look for --device-batch-size in the scripts and lower it until the run fits in memory.
- Omitting torchrun runs on one GPU with gradient accumulation, at about eight times the wait.
Use the --depth dial to change model size
nanochat is configured around one complexity setting, --depth, which is the number of transformer layers. The README says GPT-2 capability is roughly depth 26. All other hyperparameters, such as width, heads, learning rates and training horizons, are calculated automatically to keep the model compute optimal.
For quick experiments, the README describes a 12-layer model (--depth=12) that trains in about 5 minutes of pretraining, and it lists wandb metrics to watch such as val_bpb, core_metric and train/mfu.
- --depth sets the number of layers; other hyperparameters follow automatically.
- The research scripts are runs/scaling_laws.sh and runs/miniseries.sh.
OMP_NUM_THREADS=1 torchrun --standalone --nproc_per_node=8 -m scripts.base_train -- \
--depth=12 \
--run="d12" \
--model-tag="d12" \
--core-metric-every=999999 \
--sample-every=-1 \
--save-every=-1 \Run the reduced CPU or Apple Silicon path
The script runs/runcpu.sh shows a simple example for CPU or Apple Silicon. It shrinks the model so training fits in a few tens of minutes. The README is clear that this route will not give strong results.
The README also says most of the code is plain PyTorch, so it may run on other backends such as xpu, but those paths have not been fully tested by the author.
What people use it for
- Learn how tokenization, pretraining, finetuning, evaluation and inference connect in one codebase.
- Run small pretraining experiments at different --depth values to study compute-optimal model sizes.
- Reproduce a GPT-2 capability training run on a rented 8XH100 node.
- Modify the code in a readable Python and PyTorch project and re-run a short d12 experiment to see if the change helps.
- Run the reduced CPU or MPS script to explore the training pipeline without a large GPU.
Frequently asked questions
Is nanochat free to use?
The repository is released under the MIT licence, according to the listing. Running it still costs money if you use cloud GPUs; the README estimates about $48 for the GPT-2 reproduction on an 8XH100 node.
What hardware do I need to run nanochat?
The README is designed around a single 8XH100 GPU node for the GPT-2 run, and says an 8XA100 node also works, more slowly. A CUDA GPU is recommended, and a CPU or MPS path exists for a reduced run that the README says will give weak results.
How long does the nanochat speedrun take?
The README says runs/speedrun.sh takes about 1.5 hours on an 8XH100 node. It suggests running it in a screen session so the job continues if your connection drops.
Can I run nanochat on a single GPU or a Mac?
You can omit torchrun to run on one GPU, which the README says gives nearly identical results but takes about eight times longer. For Apple Silicon, use the CPU extra and runs/runcpu.sh, which trains a much smaller model with weaker results.
How does nanochat compare to nanoGPT or modded-nanogpt?
The README says the leaderboard was inspired by modded-nanogpt, and the listing names nanoGPT, llm.c and GPT-NeoX as comparable projects. The README does not compare them on features, so check each project's documentation to choose between them.
Where do I get help if something breaks?
The README points to DeepWiki for questions about the repo, the GitHub Discussions tab, and the #nanochat channel on Discord. It does not list a formal support channel.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
nanochat suits students, developers and researchers who want a small, readable codebase for a full LLM training pipeline, and who have access to multi-GPU cloud time. If you want a polished chatbot for everyday use, or you only have a CPU and expect strong results, the README says to look elsewhere.
Install options & downloads