探索的タスククラスタリングによるメタ強化学習
Meta-Reinforcement Learning via Exploratory Task Clustering
タスク間の構造的な異質性をクラスタリングで捉え、探索方策でタスク構造を発見することで、メタ強化学習のサンプル効率を改善する手法を提案した。
著者: Zhendong Chu, Hongning Wang
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Meta-reinforcement learning (meta-RL) aims to quickly solve new tasks by leveraging knowledge from prior tasks. However, previous studies often assume a single mode homogeneous task distribution, ignoring possible structured heterogeneity among tasks. Leveraging such structures can better facilitate knowledge sharing among related tasks and thus improve sample efficiency. In this paper, we explore the structured heterogeneity among tasks via clustering to improve meta-RL. We develop a dedicated exploratory policy to discover task structures via divide-and-conquer. The knowledge of the identified clusters helps to narrow the search space of task-specific information, leading to more sample efficient policy adaptation. Experiments on various MuJoCo tasks showed the proposed method can unravel cluster structures effectively in both rewards and state dynamics, proving strong advantages against a set of state-of-the-art baselines.