RadGrad: 損失勾配を用いた能動学習
RadGrad: Active learning with loss gradients
エージェントの誤差予測とリスク指標に基づいて専門家への問い合わせを選択的に行う能動学習アルゴリズムRadGradを提案し、専門家クエリ数を大幅に削減しつつDAggerと同等以上の性能をシミュレーション環境で示した。
著者: Paul Budnarain, Renato Ferreira Pinto Junior, Ilan Kogan
分類: cs.LG, cs.AI, cs.RO, stat.ML
原文アブストラクト
Solving sequential decision prediction problems, including those in imitation learning settings, requires mitigating the problem of covariate shift. The standard approach, DAgger, relies on capturing expert behaviour in all states that the agent reaches. In real-world settings, querying an expert is costly. We propose a new active learning algorithm that selectively queries the expert, based on both a prediction of agent error and a proxy for agent risk, that maintains the performance of unrestrained expert querying systems while substantially reducing the number of expert queries made. We show that our approach, RadGrad, has the potential to improve upon existing safety-aware algorithms, and matches or exceeds the performance of DAgger and variants (i.e., SafeDAgger) in one simulated environment. However, we also find that a more complex environment poses challenges not only to our proposed method, but also to existing safety-aware algorithms, which do not match the performance of DAgger in our experiments.