階層型目標条件付き方策プランニングによるマルチゴール強化学習
Proposing Hierarchical Goal-Conditioned Policy Planning in Multi-Goal Reinforcement Learning
強化学習と自動プランニングを組み合わせ、短い目標条件付き方策を階層的に整理し、MCTSで高レベル行動を計画することで、サンプル効率と推論速度を向上させる枠組みを提案した。
著者: Gavin B. Rens
分類: cs.AI, cs.LG
原文アブストラクト
Humanoid robots must master numerous tasks with sparse rewards, posing a challenge for reinforcement learning (RL). We propose a method combining RL and automated planning to address this. Our approach uses short goal-conditioned policies (GCPs) organized hierarchically, with Monte Carlo Tree Search (MCTS) planning using high-level actions (HLAs). Instead of primitive actions, the planning process generates HLAs. A single plan-tree, maintained during the agent's lifetime, holds knowledge about goal achievement. This hierarchy enhances sample efficiency and speeds up reasoning by reusing HLAs and anticipating future actions. Our Hierarchical Goal-Conditioned Policy Planning (HGCPP) framework uniquely integrates GCPs, MCTS, and hierarchical RL, potentially improving exploration and planning in complex tasks.