SAC

Example

 from rlzoo.common.env_wrappers import build_env
 from rlzoo.common.utils import call_default_params
 from rlzoo.algorithms import SAC

 AlgName = 'SAC'
 EnvName = 'Pendulum-v0'  # only continuous action
 EnvType = 'classic_control'

 # EnvName = 'BipedalWalker-v2'
 # EnvType = 'box2d'

 # EnvName = 'Ant-v2'
 # EnvType = 'mujoco'

 # EnvName = 'FetchPush-v1'
 # EnvType = 'robotics'

 # EnvName = 'FishSwim-v0'
 # EnvType = 'dm_control'

 # EnvName = 'ReachTarget'
 # EnvType = 'rlbench'

 env = build_env(EnvName, EnvType)
 alg_params, learn_params = call_default_params(env, EnvType, AlgName)
 alg = eval(AlgName+'(**alg_params)')
 alg.learn(env=env, mode='train', render=False, **learn_params)
 alg.learn(env=env, mode='test', render=True, **learn_params)

Soft Actor-Critic

class rlzoo.algorithms.sac.sac.SAC(net_list, optimizers_list, replay_buffer_capacity=500000.0)

Soft Actor-Critic

evaluate(state, epsilon=1e-06)

generate action with state for calculating gradients

get_action(state)

generate action with state for interaction with envronment

get_action_greedy(state)

generate action with state for interaction with envronment

learn(env, train_episodes=1000, test_episodes=1000, max_steps=150, batch_size=64, explore_steps=500, update_itr=3, policy_target_update_interval=3, reward_scale=1.0, save_interval=20, mode='train', AUTO_ENTROPY=True, render=False, plot_func=None)

Parameters:

• env – learning environment
• train_episodes – total number of episodes for training
• test_episodes – total number of episodes for testing
• max_steps – maximum number of steps for one episode
• batch_size – udpate batchsize
• explore_steps – for random action sampling in the beginning of training
• update_itr – repeated updates for single step
• policy_target_update_interval – delayed update for the policy network and target networks
• reward_scale – value range of reward
• save_interval – timesteps for saving the weights and plotting the results
• mode – ‘train’ or ‘test’
• AUTO_ENTROPY – automatically updating variable alpha for entropy
• render – if true, visualize the environment
• plot_func – additional function for interactive module

load_ckpt(env_name)

load trained weights

sample_action()

generate random actions for exploration

save_ckpt(env_name)

save trained weights

target_ini(net, target_net)

hard-copy update for initializing target networks

target_soft_update(net, target_net, soft_tau)

soft update the target net with Polyak averaging

update(batch_size, reward_scale=10.0, auto_entropy=True, target_entropy=-2, gamma=0.99, soft_tau=0.01)

update all networks in SAC

Default Hyper-parameters

rlzoo.algorithms.sac.default.box2d(env, default_seed=True)

rlzoo.algorithms.sac.default.classic_control(env, default_seed=True)

rlzoo.algorithms.sac.default.dm_control(env, default_seed=True)

rlzoo.algorithms.sac.default.mujoco(env, default_seed=True)

rlzoo.algorithms.sac.default.rlbench(env, default_seed=True)

rlzoo.algorithms.sac.default.robotics(env, default_seed=True)