RL Zoo requires python 3.8+ and PyTorch >= 1.13

Minimal Installation

To install RL Zoo with pip, execute:

pip install rl_zoo3

From source:

git clone
cd rl-baselines3-zoo/
pip install -e .


You can do python -m rl_zoo3.train from any folder and you have access to rl_zoo3 command line interface, for instance, rl_zoo3 train is equivalent to python

Full installation

With extra envs and test dependencies:


If you want to use Atari games, you will need to do pip install "autorom[accept-rom-license]" additionally to download the ROMs

      apt-get install swig cmake ffmpeg
      pip install -r requirements.txt
pip install -e .[plots,tests]

Please see Stable Baselines3 documentation for alternatives to install stable baselines3.

Docker Images

Build docker image (CPU):

make docker-cpu


USE_GPU=True make docker-gpu

Pull built docker image (CPU):

docker pull stablebaselines/rl-baselines3-zoo-cpu

GPU image:

docker pull stablebaselines/rl-baselines3-zoo

Run script in the docker image:

./scripts/ python --algo ppo --env CartPole-v1