深層強化学習による四足歩行ロボットのシミュレーション：歩行ポリシーへの第一歩
Quadruped Robot Simulation Using Deep Reinforcement Learning -- A step towards locomotion policy
RaiSimとPPOを用いてANYmal四足ロボットの歩行ポリシーをシミュレーション上で学習させ、限られた計算資源でも有効な制御を実現できることを示した。
著者: Nabeel Ahmad Khan Jadoon, Mongkol Ekpanyapong
分類: cs.RO
原文アブストラクト
We present a novel reinforcement learning method to train the quadruped robot in a simulated environment. The idea of controlling quadruped robots in a dynamic environment is quite challenging and my method presents the optimum policy and training scheme with limited resources and shows considerable performance. The report uses the raisimGymTorch open-source library and proprietary software RaiSim for the simulation of ANYmal robot. My approach is centered on formulating Markov decision processes using the evaluation of the robot walking scheme while training. Resulting MDPs are solved using a proximal policy optimization algorithm used in actor-critic mode and collected thousands of state transitions with a single desktop machine. This work also presents a controller scheme trained over thousands of time steps shown in a simulated environment. This work also sets the base for early-stage researchers to deploy their favorite algorithms and configurations. Keywords: Legged robots, deep reinforcement learning, quadruped robot simulation, optimal control