STL誘導スタイン変分方策勾配による疎な成功信号からのロボット方策学習
Learning Robot Policies from Sparse Success Signals via STL-Guided Stein Variational Policy Gradient
Signal Temporal Logicのロバスト性を軌道レベルの目的関数として用い、スタイン変分方策勾配で疎な成功信号からロボット方策を学習する手法を提案し、クアッドコプターとマニピュレータの6タスクで有効性を示した。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Hongrui Zheng, Cristian Ioan Vasile, Antonio Loquercio, Rahul Mangharam
分類: cs.RO
原文アブストラクト
Learning robot policies for tasks with sparse success signals is challenging when completion depends on coordinated actions, precise contact outcomes, or satisfying several conditions together. Intricate physical interactions with the world further complicate these requirements. Prior work using conventional reward shaping mechanisms provides dense feedback but local progress might not translate into eventual task completion. We present Signal Temporal Logic-guided Stein Variational Policy Gradient (STL-SVPG), a population-based method that uses smooth STL robustness as a trajectory-level training objective. Differentiating this objective through the dynamics assigns credit to policy actions according to their effect on the complete task specification, rather than local progress alone. We evaluate the approach on six quadcopter and manipulator tasks that involves event-triggered responses, strictly ordered behavior, responses within specified deadlines, and physical interaction with the world. STL-SVPG achieves the highest mean success rate among the compared methods on five of six benchmarks. Simulation-trained policies trained in simulation transfer temporal and contact task behavior to the real world.