多様性の追求：深層強化学習エージェントの多目的テスト
The Pursuit of Diversity: Multi-Objective Testing of Deep Reinforcement Learning Agents
深層強化学習エージェントの安全性テストにおいて、失敗の多様性と発生しやすさを同時に最適化する多目的探索手法INDAGO-Nexusを提案し、従来手法より多くの固有の失敗を発見した。
著者: Antony Bartlett, Cynthia Liem, Annibale Panichella
分類: cs.LG
原文アブストラクト
Testing deep reinforcement learning (DRL) agents in safety-critical domains requires discovering diverse failure scenarios. Existing tools such as INDAGO rely on single-objective optimization focused solely on maximizing failure counts, but this does not ensure discovered scenarios are diverse or reveal distinct error types. We introduce INDAGO-Nexus, a multi-objective search approach that jointly optimizes for failure likelihood and test scenario diversity using multi-objective evolutionary algorithms with multiple diversity metrics and Pareto front selection strategies. We evaluated INDAGO-Nexus on three DRL agents: humanoid walker, self-driving car, and parking agent. On average, INDAGO-Nexus discovers up to 83% and 40% more unique failures (test effectiveness) than INDAGO in the SDC and Parking scenarios, respectively, while reducing time-to-failure by up to 67% across all agents.