実行フィードバックによる価値と方策のキャリブレーションで社会的に調和する自動運転
Value and Policy Calibration from Execution Feedback for Socially Compatible Driving
ワールドモデル強化学習で、想像上の相互作用と実際の走行フィードバックを結びつけ、自車の進行と周囲の人間運転車の負担を両立させる交差点意思決定手法を提案した。
著者: Sen Shen, Ning Liu, Xiangrui Kong, Yingwei Li, Dongkun Han
分類: cs.RO, cs.AI
原文アブストラクト
Autonomous vehicle (AV) decisions at intersections affect the delay, braking, and motion smoothness of surrounding human-driven vehicles (HDVs). Learning socially compatible behavior requires connecting imagined interaction outcomes with feedback from executed actions. We propose a world-model reinforcement learning framework combines imagined policy improvement with direct value and policy calibration from finalized interactions, balancing AV progress with HDV interaction burden. An interaction-aware latent model supports imagination, while off-policy correction and replay updates enable observed rewards and transitions to supervise both the critic and actor. Simulation experiments at an unsignalized intersection achieve a 100% success and reductions in following-HDV acceleration variation of 65.5% and 44.0% relative to PIWM and DreamerV3, respectively. Ablations identify complementary roles for value calibration in reducing HDV burden and policy calibration in improving passage efficiency. Tests under out-of-distribution traffic shifts further show lower ego jerk and HDV acceleration variation than PIWM at comparable passage times.
