Stable Baselines3 vs Tianshou
Which should you use? A side-by-side look at what each one does, who it suits, what it costs, how hard it is to set up, and where each falls short.
Open source · MIT
A Python library of reliable reinforcement learning algorithms built on PyTorch.
Open source · MIT
A Python library for building and training reinforcement learning agents using PyTorch.
The short answer
Choose Stable Baselines3 if you need…
- Researchers who want a reliable baseline to compare a new method against
- Engineers prototyping an RL agent for a task with a Gymnasium-style environment
- Students with some RL background who want readable, tested implementations
- Users who want to train PPO on a simple environment such as CartPole with a few lines of code
Choose Tianshou if you need…
- Researchers who want to implement or modify reinforcement learning algorithms in PyTorch
- Practitioners training a supported algorithm on their own Gymnasium-style environment
- Students studying how common RL algorithms are implemented
- Developers who want vectorized or parallel environment sampling
Side-by-side
| Stable Baselines3 | Tianshou | |
|---|---|---|
| Type | Library | Library |
| Price | Free and open source | Free and open source |
| Licence | MIT | MIT |
| GitHub stars | 14k | 11k |
| Setup difficulty | Intermediate | Advanced |
| How you get it | Install with a package manager | Install with a package manager |
| Main language | Python | Python |
| Last code update | 11 days ago | 7 months ago |
What each one is
Stable Baselines3
Stable Baselines3 (SB3) is a Python library that provides tested implementations of common reinforcement learning algorithms, which are methods for training software agents to make decisions by trial and reward. It is written in PyTorch and offers a consistent interface so researchers and engineers can train, compare, and build on these algorithms without writing them from scratch. The README notes that it assumes some prior knowledge of reinforcement learning.
Tianshou
Tianshou is a reinforcement learning library built on PyTorch and Gymnasium. It provides many ready-made learning algorithms, such as DQN, PPO, SAC and TD3, so you can train agents that learn by interacting with an environment. It offers both high-level interfaces for applying existing algorithms and lower-level interfaces for researchers writing new ones.
Best for
Stable Baselines3
- Researchers who want a reliable baseline to compare a new method against
- Engineers prototyping an RL agent for a task with a Gymnasium-style environment
- Students with some RL background who want readable, tested implementations
- Users who want to train PPO on a simple environment such as CartPole with a few lines of code
Tianshou
- Researchers who want to implement or modify reinforcement learning algorithms in PyTorch
- Practitioners training a supported algorithm on their own Gymnasium-style environment
- Students studying how common RL algorithms are implemented
- Developers who want vectorized or parallel environment sampling
Strengths
Stable Baselines3
- Provides a common, consistent interface across algorithms, as described in the feature table
- Includes documentation, custom environment and policy support, TensorBoard support, and type hints per the README
- Has an active CI pipeline and is documented as a successor to Stable Baselines
Tianshou
- Covers a broad set of algorithms across online, offline and imitation learning
- Offers both high-level and procedural APIs for different levels of control
- Supports vectorized environments, recurrent networks, multi-GPU training and common logging tools
Not ideal for
Stable Baselines3
- Complete beginners with no reinforcement learning background, since the README says the library assumes some RL knowledge
- Projects that need the newest experimental algorithms, which the README says are placed in the separate SB3-Contrib repository
- Users who need the fastest possible training, since the README mentions the separate SBX (JAX) variant as faster but with fewer features
Tianshou
- People new to reinforcement learning who want a guided, beginner-focused course (the README is a library reference, not a tutorial)
- Anyone needing a non-Python stack, since the project is Python-based and requires Python 3.11 or newer
- Users who need a fully stable, long-term API without changes, since version 2 broke compatibility with earlier versions
Trade-offs
Stable Baselines3
- The README says development is now focused on bug fixes and maintenance, with no major changes planned
- Requires some reinforcement learning knowledge to use effectively
- Newer or experimental features live in separate repositories, so a full toolset means installing several projects
Tianshou
- Version 2 is not backwards compatible with earlier versions, so older code and tutorials may need migration
- Multi-agent and model-based RL are described only as experimental support
- Installation is recommended via cloning and using poetry, which adds setup steps
Getting started
Stable Baselines3
Install with optional extras (Tensorboard, OpenCV, ale-py, pandas, matplotlib)
pip install 'stable-baselines3[extra]'Install core package without extras
pip install stable-baselines3Tianshou
Install from PyPI
pip install tianshouClone the repository
git clone https://github.com/thu-ml/tianshou.gitFrequently asked questions
What is the difference between Stable Baselines3 and Tianshou?
Stable Baselines3: A Python library of reliable reinforcement learning algorithms built on PyTorch. Tianshou: A Python library for building and training reinforcement learning agents using PyTorch. Stable Baselines3 suits researchers who want a reliable baseline to compare a new method against, while Tianshou suits researchers who want to implement or modify reinforcement learning algorithms in pytorch.
Are Stable Baselines3 and Tianshou free?
Stable Baselines3: Free and open source. Tianshou: Free and open source.
Which is easier to set up, Stable Baselines3 or Tianshou?
Stable Baselines3 is easier to get started with (rated intermediate); Tianshou is rated advanced and usually needs more setup.
Is Stable Baselines3 or Tianshou more popular?
Stable Baselines3 has more GitHub stars (14k) than Tianshou (11k). Popularity isn't the same as fit — check which one matches your use case above.
What are other alternatives to Stable Baselines3 and Tianshou?
Other options people compare them with include Gymnasium, Weights & Biases, ONNX Runtime, Petals.
Keep comparing
Built from our catalogue entries for both products: open-source details come from each project's README (AI-assisted, refreshed weekly) and GitHub data; AI-tool details are editorial. Spotted something wrong? Let us know from either product's page.