Baselines
Reference implementations of reinforcement learning algorithms in Python, for research comparison.
- Stars
- 16.8k
- Forks
- 4.9k
- Licence
- MIT
- Language
- Python
- Last update
- 2.3 years ago
- Type
- Library
No commits in over a year — check for forks or alternatives before adopting.
What it is
OpenAI Baselines is a collection of reinforcement learning algorithms (such as DQN, PPO and A2C) implemented in Python. Researchers use it to reproduce published results, test new ideas on top of a known starting point, and compare new methods against established ones. The README says the status is maintenance, meaning only bug fixes and minor updates are expected.
- Train a PPO2 agent on a MuJoCo Humanoid environment with the baselines.run command
- Train a DQN agent to play Atari Pong using the deepq algorithm
- Save a trained model with --save_path and reload it later to watch it play
- Log training progress and metrics to a chosen folder with --log_path
Great for
- Researchers who want a trusted baseline to compare a new reinforcement learning method against
- Students learning how classic RL algorithms are implemented
- Developers who want to train an RL agent on MuJoCo or Atari environments with existing code
- People reproducing published reinforcement learning benchmark results
Not ideal for
- Anyone needing current TensorFlow 2 support, since the master branch supports TensorFlow 1.4 to 1.14 and the README points to a tf2 branch for 2.0 instead
- Projects that need actively developed code, since the README says the project is in maintenance mode
- Users who need MuJoCo out of the box, since MuJoCo is proprietary and requires a license for some examples
How to get it
- 1
Clone and build from source
You'll need: Python 3.5 or newer with development headers · CMake · OpenMPI · zlib · TensorFlow 1.14 (per the README)
Install system packages on Ubuntu
sudo apt-get update && sudo apt-get install cmake libopenmpi-dev python3-dev zlib1g-devClone the repository
git clone https://github.com/openai/baselines.gitInstall TensorFlow 1.14
pip install tensorflow==1.14Install the baselines package
pip install -e . - 2
Run it
- Install the system packages and TensorFlow 1.14 as described in the README
- Run the test suite with pytest to confirm the installation
- Train an example: python -m baselines.run --alg=deepq --env=PongNoFrameskip-v4 --num_timesteps=1e6
- 3
Get the most out of it
- Use --save_path and --load_path to save a trained model and reload it to visualize results
- Set --log_path to a unique folder so logs are not overwritten
- Adjust hyperparameters such as --ent_coef or --num_layers from the command line
- Review the benchmarks pages for MuJoCo and Atari to compare your results
Downloads
Strengths
- Implementations are described as roughly on par with published paper scores for DQN and its variants
- Provides several standard algorithms in one consistent codebase with a single command-line entry point
- Includes a benchmarks page for MuJoCo and Atari results
Trade-offs
- Built for older TensorFlow 1.x versions, so setup on modern Python and TensorFlow environments may be difficult
- The README says the model saving and loading API is not yet unified across algorithms
How it compares
| Project | Stars | Type | Level | Updated | Best for |
|---|---|---|---|---|---|
| baselines (this) | 16.8k | Library | intermediate | 2.3 years ago | Researchers who want a trusted baseline to compare a new reinforcement learning method against |
| stable-baselines3 | 14k | Library | intermediate | 11 days ago | Researchers who want a reliable baseline to compare a new method against |
Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.