Guide · Open source

How to use Gymnasium: install, run environments and tips

Updated 2026-10-29 · written with AI from the project's own documentation and checked against our catalogue data

Gymnasium is an open-source Python library that gives reinforcement learning algorithms and simulated environments a standard way to talk to each other. It also ships a set of ready-made environments, from classic control problems to Atari games. This guide shows you how to install it, create an environment, run an interaction loop and know when it is the right tool.

GymnasiumFarama-Foundation/Gymnasium · ★ 12.8k · MIT Install options & downloads

Quick start

1Install the library. Run pip install gymnasium in your Python environment. Add the extras for the environment family you need.
2Create an environment. Call gym.make with a name such as CartPole-v1.
3Reset the environment. Call env.reset with a seed to start an episode and get the first observation.
4Step through the episode. Choose an action, call env.step, and read the observation, reward, terminated and truncated values.
5Reset when an episode ends. If terminated or truncated is true, call env.reset again.
6Close the environment. Call env.close() when you finish.

Install Gymnasium with pip

Gymnasium is installed with pip, Python's package manager. Install the base library first. The base install does not include the dependencies for every environment family, because there are many of them and some are difficult to install on certain systems.

If you want one environment family, install its dependencies by name. For all families, use the 'all' option.

Base library only: pip install gymnasium
One family (example: Atari): pip install "gymnasium[atari]"
All families: pip install "gymnasium[all]"

pip install gymnasium
pip install "gymnasium[atari]"
pip install "gymnasium[all]"

Choose an environment family

Gymnasium groups its environments into families. Classic Control contains classic reinforcement learning problems based on real-world physics. Box2D contains physics-control toy games that use Box2D physics and PyGame rendering. Toy Text contains very simple environments with small discrete state and action spaces, which makes them useful for debugging an implementation.

MuJoCo contains multi-joint control environments that are more complex than Box2D. Atari emulates Atari 2600 ROMs with a wide range of difficulty. A third-party category lists environments built for the Gymnasium API; the README advises checking which version each one was made for and using apply_env_compatibility in gymnasium.make if needed.

Classic Control: classic physics-based problems
Box2D: physics-control toy games
Toy Text: small, easy environments for debugging
MuJoCo: complex multi-joint control
Atari: Atari 2600 games
Third-party: community environments that follow the API

Create an environment and run a loop

Every Gymnasium environment is created with gym.make and a name such as CartPole-v1. Call env.reset to start an episode, which returns an observation and an info dictionary. Pass a seed to reset to make runs reproducible.

Then repeat env.step with an action until the episode ends. The step call returns an observation, a reward, terminated and truncated flags, and info. When either flag is true, reset the environment. The example below uses random actions from action_space.sample(), so it shows the loop rather than a trained agent. Call env.close() when finished.

import gymnasium as gym
env = gym.make("CartPole-v1")

observation, info = env.reset(seed=42)
for _ in range(1000):
    action = env.action_space.sample()
    observation, reward, terminated, truncated, info = env.step(action)

    if terminated or truncated:
        observation, info = env.reset()
env.close()

Plug in your own learning algorithm

The point of the shared API is that your agent code does not change when the environment changes. Your algorithm reads the observation, chooses an action, and uses the reward it gets back. Replace the random sample() call with your policy to train an agent.

Several learning libraries already support the Gymnasium API. The README names CleanRL as a library built on it, aimed at newer people in the field and offering reference implementations. It also notes that PettingZoo is the multi-agent counterpart of Gymnasium.

Keep the reset and step loop the same.
Replace action_space.sample() with your policy.
Use the same environment names to compare algorithms fairly.

Keep experiments reproducible

Gymnasium uses strict versioning for reproducibility. Every environment name ends in a suffix such as -v0 or -v1. When a change could affect learning results, the number goes up by one, so older results are not silently compared with new ones.

Pass a seed to env.reset to get repeatable episodes. Note the exact environment name, including its version suffix, when you report results.

Record the full environment name, for example CartPole-v1.
Set a seed in env.reset(seed=...).

Migrating from OpenAI Gym

Gymnasium is a fork of OpenAI's Gym library. The maintainers of the fork took over maintenance after OpenAI handed it to an outside team, and future maintenance happens in Gymnasium. If you have older Gym code, it is worth moving to Gymnasium so it keeps receiving updates.

The README does not list a step-by-step migration guide, so check the documentation at gymnasium.farama.org for the changes that matter to your code.

What people use it for

Comparing reinforcement learning algorithms on the same standard environments.
Learning reinforcement learning with simple, well-known test problems such as CartPole.
Debugging a new algorithm on Toy Text environments, which have small state and action spaces.
Training agents on physics-control problems from Box2D or MuJoCo.
Running agents on Atari games through the Atari family.

Frequently asked questions

Is Gymnasium free to use?

Yes. Gymnasium is open source under the MIT licence, and its source is on GitHub. The README also invites people who can afford it to support development through donations.

What do I need to run Gymnasium?

You need Python and pip. The base library is installed with pip install gymnasium. Some environment families need extra dependencies, which you install with the matching extra, such as gymnasium[atari].

Why does pip install gymnasium not give me Atari or MuJoCo?

The base install leaves out the dependencies for most environment families, because there are many and some are hard to install on certain systems. Install the family you need with pip install "gymnasium[atari]" or install everything with pip install "gymnasium[all]".

Is Gymnasium the same as OpenAI Gym?

No, but it is closely related. Gymnasium is a maintained fork of OpenAI's Gym library, and the maintainers of the fork now handle future development. Environment names inherited from Gym keep the same -v suffix system.

Can I use Gymnasium for multi-agent reinforcement learning?

The README says Gymnasium covers single-agent setups. For multi-agent work it points to PettingZoo, which it describes as a multi-agent version of Gymnasium.

What alternatives are there to Gymnasium?

The README lists CleanRL, a learning library built on the Gymnasium API, and PettingZoo for multi-agent problems. The project's comparable projects list also names OpenAI Gym, Stable-Baselines3 and CleanRL. Choose based on whether you need an environment API, a learning library or multi-agent support.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Gymnasium suits researchers, students and developers who want a shared, maintained environment API and standard test problems to build and compare reinforcement learning code on. It is a library for experiments rather than a finished product, and it is Python-only and single-agent, so people who need a ready-made application or multi-agent environments should look at other options such as PettingZoo.

Install options & downloads