変化点対応ワールドモデル：動力学シフトの検出と古いリプレイの忘却による回復
Changepoint-Aware World Models: Detecting Dynamics Shifts and Recovering by Forgetting Stale Replay in Model-Based RL
モデルベース強化学習エージェントが自身の予測誤差から動力学の急変を検出し、古いリプレイバッファを忘却することで、環境変化後の適応を高速化する手法を提案。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Everest Yang
分類: cs.LG
原文アブストラクト
A robot's learned model of its own dynamics is only valid until those dynamics change: actuators wear, payloads shift, and joints stiffen. A model-based agent that keeps training as if nothing happened adapts slowly, dragged back by a replay buffer full of stale experience. We present Changepoint-Aware World Models (CAWM), a DreamerV3 agent that detects an abrupt dynamics shift from its own internal prediction error, using an online CUSUM test against a rolling baseline that fires only on abrupt change rather than on slow learning drift. It then forgets stale replay, keeping the learned representation while flushing obsolete data. On simulated locomotion under two robot-relevant shifts, doubled gravity and halved actuator gain, CAWM recovers substantially faster than passive retraining. It also beats a strong baseline that respawns a fresh dynamics model on detection, the deep-world-model analogue of model-bank methods. With the response triggered at the shift, CAWM gains +95 to +153 return in the first 30k post-shift frames over three seeds, while matching that respawn at asymptote. Running the detector in closed loop reproduces this gain on the gravity shift. The benefit holds across both shift types, and is largest when the shift is severe enough that old data is genuinely obsolete.