Configurations Overview

Supported DRL Agorithms

Generally RLzoo supports following DRL algorithms:

Value-based methods

• Deep Q-Networks (DQN)
• Double DQN
• Dueling DQN
• Prioritized Experience Replay (PER)
• Retrace
• Noisy DQN
• Distributed DQN

Policy-based methods

• Vanilla Policy Gradient (VPG)
• Trust Region Policy Optimization (TRPO)
• Proximal Policy Optimization (PPO)
• Distributed PPO (DPPO)

Actor-critic methods

• Actor-Critic (AC)
• Asychronous Advantage Actor-Critic (A3C)
• Deep Deterministic Policy Gradient (DDPG)
• Twin Delayed DDPG (TD3)
• Soft Actor-Critic (SAC)

Supported Environments

Generally RLzoo supports following environments for DRL:

• OpenAI Gym

  • Atari
  • Box2D
  • Classic Control
  • MuJoCo
  • Robotics

• DeepMind Control Suite
• RLBench

Full list of specific names of environments supported in RLzoo can be checked in List of Supported Environments in RLzoo.

Supported Configurations

Not all configurations (specific RL algorithm on specific environment) are supported in RLzoo, as in other libraries. The supported configurations for RL algorithms with corresponding environments in RLzoo are listed in the following table.

Algorithms	Action Space	Policy	Update	Envs
DQN (double, dueling, PER)	Discrete Only	NA	Off-policy	Atari, Classic Control
AC	Discrete/Continuous	Stochastic	On-policy	All
PG	Discrete/Continuous	Stochastic	On-policy	All
DDPG	Continuous	Deterministic	Off-policy	Classic Control, Box2D, MuJoCo, Robotics, DeepMind Control, RLBench
TD3	Continuous	Deterministic	Off-policy	Classic Control, Box2D, MuJoCo, Robotics, DeepMind Control, RLBench
SAC	Continuous	Stochastic	Off-policy	Classic Control, Box2D, MuJoCo, Robotics, DeepMind Control, RLBench
A3C	Discrete/Continuous	Stochastic	On-policy	Atari, Classic Control, Box2D, MuJoCo, Robotics, DeepMind Control
PPO	Discrete/Continuous	Stochastic	On-policy	All
DPPO	Discrete/Continuous	Stochastic	On-policy	Atari, Classic Control, Box2D, MuJoCo, Robotics, DeepMind Control
TRPO	Discrete/Continuous	Stochastic	On-policy	All