DADAgger: 不一致拡張データセット集約
DADAgger: Disagreement-Augmented Dataset Aggregation
模倣学習のDAggerを改良し、アンサンブルの予測分散で分布外状態のみを専門家に問い合わせることで、専門家クエリを削減しつつ同等の性能を達成する手法を提案。
著者: Akash Haridas, Karim Hamadeh, Samarendra Chandan Bindu Dash
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
DAgger is an imitation algorithm that aggregates its original datasets by querying the expert on all samples encountered during training. In order to reduce the number of samples queried, we propose a modification to DAgger, known as DADAgger, which only queries the expert for state-action pairs that are out of distribution (OOD). OOD states are identified by measuring the variance of the action predictions of an ensemble of models on each state, which we simulate using dropout. Testing on the Car Racing and Half Cheetah environments achieves comparable performance to DAgger but with reduced expert queries, and better performance than a random sampling baseline. We also show that our algorithm may be used to build efficient, well-balanced training datasets by running with no initial data and only querying the expert to resolve uncertainty.