Guide · Open source
How to use reinforcement-learning: clone, run notebooks and study RL algorithms
Updated · written with AI from the project's own documentation and checked against our catalogue data
dennybritz/reinforcement-learning is an MIT-licensed collection of Jupyter notebooks that implement classic reinforcement learning algorithms in Python 3, using OpenAI Gym environments. It is built as a study aid: each folder pairs exercises and worked solutions with Sutton and Barto's textbook and David Silver's lecture course. This guide shows you how to get the code running and how to pick a path through the material.
reinforcement-learningdennybritz/reinforcement-learning · ★ 22.2k · MIT Install options & downloads Quick start
- 1Install prerequisites. Make sure Python 3 and Jupyter Notebook are installed on your machine.
- 2Clone the repository. Run git clone https://github.com/dennybritz/reinforcement-learning to download the code.
- 3Read the textbook chapter. Open the folder that matches your chapter and read its learning goals and concept summary.
- 4Open the exercise notebook. Start the exercise notebook in Jupyter and attempt the problems before checking answers.
- 5Compare with the solution. Open the matching solution notebook, such as TD/Q-Learning Solution.ipynb, and compare results.
- 6Move to advanced topics. Once tabular methods make sense, try the function approximation and deep Q-learning notebooks, noting the WIP entries.
What you need before you start
The repository is set up as a clone-and-build project. The README lists Python 3 and Jupyter Notebook as the prerequisites, and all code is written in Python 3 with RL environments from OpenAI Gym.
Some advanced parts use TensorFlow for neural network implementations, so you will need it only if you plan to run those notebooks.
- Python 3
- Jupyter Notebook
- OpenAI Gym (used by the environments in the notebooks)
- TensorFlow (only for the advanced deep Q-learning and policy gradient notebooks)
Get the code with git clone
Copy the repository to your machine with git. The README gives this command as the way to start.
git clone https://github.com/dennybritz/reinforcement-learningOpen the notebooks and follow the chapter folders
Each folder corresponds to one or more chapters of the textbook or course. Folders also include learning goals, a brief concept summary and links to readings, so you can read the theory alongside the code.
The table of contents runs from an introduction to OpenAI Gym, through MDPs and Bellman equations, dynamic programming, Monte Carlo methods, temporal difference learning, and function approximation. Deep Q-learning and policy gradient methods follow, and several of those entries are marked WIP in the README.
- Introduction to RL problems and OpenAI Gym
- MDPs and Bellman Equations
- Dynamic Programming: policy iteration and value iteration
- Monte Carlo model-free prediction and control
- Temporal Difference model-free prediction and control
- Function Approximation
- Deep Q Learning (WIP)
- Policy Gradient Methods (WIP)
Run tabular methods like Q-learning and SARSA
The tabular methods are the best place for a first run. The README lists SARSA (on-policy TD learning) and Q-learning (off-policy TD learning) as implemented notebooks. Both are in the TD folder.
Work through the exercise notebook first, then compare it with the solution notebook. The README lists the solution files by name, so you can find them directly.
- SARSA (On Policy TD Learning): TD/SARSA Solution.ipynb
- Q-Learning (Off Policy TD Learning): TD/Q-Learning Solution.ipynb
- Dynamic Programming Value Iteration: DP/Value Iteration Solution.ipynb
Use the repo alongside the textbook or David Silver's course
The materials are designed to complement Reinforcement Learning: An Introduction (2nd Edition) by Sutton and Barto and David Silver's Reinforcement Learning Course. Read the matching chapter, then run the folder's exercises.
The README also lists other courses and talks, and further papers on deep RL, if you want to go beyond the repo.
- Textbook: Reinforcement Learning: An Introduction (2nd Edition)
- Course: David Silver's Reinforcement Learning Course (UCL)
- Also listed: OpenAI Spinning Up, CS294-112 Deep Reinforcement Learning (UC Berkeley)
Know the limits before you build on it
The README frames this repository as a learning aid, not a production library. Treat its code as examples to read and modify. If you need a maintained toolkit for real projects, look at the comparable projects below.
Comparable projects listed in the source are openai/spinningup, keras-rl and carpedm20/deep-rl-tensorflow.
- Learning aid, not a production RL library
- Some deep RL entries are WIP (for example Prioritized Experience Replay, DDPG)
- Comparable: openai/spinningup, keras-rl, carpedm20/deep-rl-tensorflow
What people use it for
- Working through Sutton and Barto's textbook chapter by chapter with runnable code
- Implementing and comparing Q-learning and SARSA on OpenAI Gym environments
- Checking your own dynamic programming or Monte Carlo solutions against worked notebooks
- Following David Silver's lectures with matching exercises
- Reading how an actor-critic or REINFORCE method is coded in a notebook
Frequently asked questions
Is dennybritz reinforcement-learning free to use?
The repository is licensed under MIT, as listed in the source. You can use, copy and modify it under that licence; check the LICENSE file in the repo for the full terms.
What do I need to install to run it?
The README lists Python 3 and Jupyter Notebook as prerequisites. The code uses OpenAI Gym environments, and advanced parts use TensorFlow.
How do I download the reinforcement-learning code?
Run git clone https://github.com/dennybritz/reinforcement-learning in a terminal. Then open the notebooks in Jupyter.
Does it work with Sutton and Barto's book?
Yes. Each folder maps to one or more chapters of Reinforcement Learning: An Introduction (2nd Edition), and includes learning goals and links to readings.
Is this a production reinforcement learning library?
No. The README frames it as a learning tool. For production work, the comparable projects in the source, such as openai/spinningup or keras-rl, are alternatives to consider.
Which parts are not finished?
Several entries are marked WIP in the README, including Deep Q-Learning with Prioritized Experience Replay, Deterministic Policy Gradients, and DDPG. Check the README for the current status.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this repository if you are studying reinforcement learning with Sutton and Barto or David Silver's course and want runnable Python notebooks for tabular methods like Q-learning and SARSA. Pick something else, such as openai/spinningup or keras-rl, if you need production-ready libraries or the latest deep RL methods, which several entries here mark as work in progress.
Install options & downloads