minGPT
A small, readable PyTorch re-implementation of GPT for training and running language models.
- Stars
- 25k
- Forks
- 3.3k
- Licence
- MIT
- Language
- Python
- Last update
- 2.2 years ago
- Type
- Library
No commits in over a year — check for forks or alternatives before adopting.
What it is
minGPT is a compact PyTorch codebase that reimplements the GPT transformer model, covering both training and inference. The README describes it as roughly 300 lines of model code, split into a model file, a byte pair encoder, and a generic training loop, aimed at being easy to read and learn from. It is a library you import into your own Python project, with demos for tasks like addition and character-level text generation.
- Instantiate a GPT-2 (124M parameter) configuration from the model file
- Train a small character-level model on your own text file using projects/chargpt
- Run the projects/adder demo to see a GPT learn to add numbers
- Step through demo.ipynb to see a minimal GPT and Trainer on a sorting example
- Load a pretrained GPT-2 and generate text using generate.ipynb
Great for
- Students who want to read a working GPT implementation end to end
- Developers who want a small, hackable base for experimenting with transformer models
- People who want to load GPT-2 weights and generate text with PyTorch
- Researchers who prefer a minimal codebase over a large framework
Not ideal for
- People who need an actively maintained project: the README says minGPT is in a semi-archived state and points to nanoGPT for newer work
- Anyone needing distributed training or mixed precision, which the README lists as todos
- Users looking for a ready-made chatbot or polished end-user app
How to get it
- 1
Clone and build from source
You'll need: Python · PyTorch
Clone the repository
git clone https://github.com/karpathy/minGPT.gitEnter the directory
cd minGPTInstall as a library in editable mode
pip install -e . - 2
Run it
- Install the library with pip install -e . from the cloned folder
- Import the model with from mingpt.model import GPT
- Open demo.ipynb to see a minimal training example on a sorting task
- Run python -m unittest discover tests to check the unit tests
- 3
Get the most out of it
- Read mingpt/model.py first, since the README says the whole model is about 300 lines
- Use the GPT.get_default_config() and Trainer.get_default_config() helpers to set options
- Try projects/chargpt on your own text file to see character-level generation
- Use generate.ipynb to load pretrained GPT-2 weights and experiment with prompts
- Check the README references for the GPT-1, GPT-2, and GPT-3 papers alongside the code
Downloads
Strengths
- Small enough to read in an afternoon, with the model in about 300 lines according to the README
- Clear separation between model, tokenizer, and training loop
- MIT licensed and widely referenced in notebooks, blogs, and courses
Trade-offs
- Semi-archived as of January 2023; the author suggests nanoGPT for newer development
- Test coverage is described as limited, and there is no requirements.txt yet per the README
- Lacks mixed precision and distributed training, which are listed as future todos
How it compares
| Project | Stars | Type | Level | Updated | Best for |
|---|---|---|---|---|---|
| minGPT (this) | 25k | Library | intermediate | 2.2 years ago | Students who want to read a working GPT implementation end to end |
| nanoGPT | 64.4k | Library | intermediate | 12 months ago | Developers who want to read a compact GPT training implementation |
| llm.c | 31.4k | Learning resource | advanced | 1.4 years ago | Developers who want to see how a GPT-2 style model is trained at the lowest level |
Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.