物体のピック&プレース学習のための報酬設計
Reward Engineering for Object Pick and Place Training
強化学習で物体把持を行う際、エンドエフェクタと目標のxyz軸距離に重み付けする報酬設計により学習時間をほぼ半減させ、さらにマンハッタン軌道など所望の軌道を学習させることに成功した。
著者: Raghav Nagpal, Achyuthan Unni Krishnan, Hanshen Yu
分類: cs.AI, cs.RO
原文アブストラクト
Robotic grasping is a crucial area of research as it can result in the acceleration of the automation of several Industries utilizing robots ranging from manufacturing to healthcare. Reinforcement learning is the field of study where an agent learns a policy to execute an action by exploring and exploiting rewards from an environment. Reinforcement learning can thus be used by the agent to learn how to execute a certain task, in our case grasping an object. We have used the Pick and Place environment provided by OpenAI's Gym to engineer rewards. Hindsight Experience Replay (HER) has shown promising results with problems having a sparse reward. In the default configuration of the OpenAI baseline and environment the reward function is calculated using the distance between the target location and the robot end-effector. By weighting the cost based on the distance of the end-effector from the goal in the x,y and z-axes we were able to almost halve the learning time compared to the baselines provided by OpenAI, an intuitive strategy that further reduced learning time. In this project, we were also able to introduce certain user desired trajectories in the learnt policies (city-block / Manhattan trajectories). This helps us understand that by engineering the rewards we can tune the agent to learn policies in a certain way even if it might not be the most optimal but is the desired manner.
関連論文
- 環境に寄りかかって遠くの物体に手を伸ばすヒューマノイド制御マニピュレーション
- 目的関数をポリシーとして:接触を伴うマニピュレーションのための視覚言語モデルによるコストプログラムマニピュレーション
- 見えない開始遅延が自己対戦ハンドオーバーを未知のパートナーに頑健にするマニピュレーション
- 非線形マニピュレータの単一線形化軌道計画における最適動作点選択マニピュレーション
- VRヒューマンデモで点群把持ポリシーの持続的失敗を修正マニピュレーション
- 低出力二重振り子の自然な振り上げ制御マニピュレーション
