gym 搭建 RL 環境

gym調用

gym的調用遵從以下的順序

1. env = gym.make('x')
2. observation = env.reset()
3. for i in range(time_steps):
   env.render()
   action = policy(observation)
   observation, reward, done, info = env.step(action)
   if done:
   ……
   break
4. env.close()

例程

例程是一個簡單的策略，桿左斜車左移，右斜則右移。

import gym
import numpy as np
env = gym.make('CartPole-v0')
t_all = []
action_bef = 0
for i_episode in range(5):
    observation = env.reset()
    for t in range(100):
        env.render()
        cp, cv, pa, pv = observation
        if abs(pa)<= 0.1:
            action = 1 -action_bef
        elif pa >= 0:
            action = 1
        elif pa <= 0:
            action = 0
        observation, reward, done, info = env.step(action)
        action_bef = action
        if done:
            # print("Episode finished after {} timesteps".format(t+1))
            t_all.append(t)
            break
        if t ==99:
            t_all.append(0)
env.close()
print(t_all)
print(np.mean(t_all))

gym的搭建

gym的函數構成

一個完整的gym環境包括以下函數：類構建、初始化、

• class Cartpoleenv(gym.env)
  • def __ init __(self):
  • def reset(self):
  • def seed(self, seed = None): return [seed]
  • def step(self, action): return self.state, reward, done, {}
  • def render(self, mode='human'): return self.viewer.render()
  • def close():

功能函數

• 參數限位 vel = np.clip(vel, vel_min, vel_max)

• action輸入校驗
  self.action_space.contains(action)

• action和observation空間定義
  Discrete: 0,1,2
  low = np.array([min_0,min_1],dtype=np.float32)
  high = np.array([max_0,max_1],dtype=np.float32)

  self.action_space = spaces.Discrete(3)
  self.observation_space = spaces.Box(
  
  self.low, self.high, dtype=np.float32)
  

Mac系統添加自己寫的環境到gym

1. 打開gym.envs目錄：/usr/local/lib/python3.7/site-packages/gym/envs
2. 將自己編寫的myenv.py拷貝至一個aa目錄
3. envs/aa下__init__.py添加 from gym.envs.classic_control.myenv import MyEnv
4. env下__init__.py添加

register(
id='myenv-v0',
entry_point='gym.envs.classic_control:MyEnv,
max_episode_steps=999, 
)

• 注意：注冊方法內 id號不能省
  然后就可以調用了

id = 'myenv-v0'
env = gym.make('id')
env.reset()
env.step()
env.sloce()