行動結果フィードバックによるテスト時方策適応フレームワークSCOUT
Scouting the Dynamics Gap: Test-Time Policy Adaptation via Action-Outcome Feedback
行動予測方策と順ダイナミクスモデルを共有信念潜在空間で結合し、行動と結果の不一致から環境ダイナミクスをオンラインで推定・適応するメタ学習手法を提案。シミュレーションと実機で適応速度とsim-to-real転移の堅牢性を示した。
著者: Yishu Li, Liyuan Geng, Xinyi Mao, Amber Li, David Held
分類: cs.RO
原文アブストラクト
While pretrained robotic policies exhibit impressive capabilities in controlled environments, unobserved physical properties and dynamics require these policies to rapidly adapt during deployment. Existing test-time adaptation methods typically rely on sparse scalar rewards, failing to exploit the rich geometric and dynamic feedback from the environment during physical interaction. To address this challenge, we propose SCOUT, a dynamics-aware meta-learning framework that enables manipulation policies to rapidly adapt by continuously revising their internal beliefs about environment dynamics. Our approach couples an action-prediction policy with a forward dynamics model via a shared belief latent space. During meta-training, an inner loop updates this shared belief latent by minimizing the dynamics prediction error against the observed action outcome, while the outer loop optimizes the network for action selection. At deployment, this structure allows the agent to infer and adapt to unknown physical dynamics on the fly. By updating its latent belief based on action-outcome mismatches, the policy automatically adapts without risking catastrophic forgetting. We demonstrate that SCOUT significantly accelerates online adaptation across simulated manipulation benchmarks and achieves robust sim-to-real transfer in the real world. Project webiste can be found here: https://liy1shu.github.io/SCOUT/