FAR: 失敗を考慮した再試行によるテスト時回復と継続的ポリシー改善
FAR: Failure-Aware Retry for Test-Time Recovery and Continual Policy Improvement
ロボットがテスト時に失敗から学習し、行動を適応させて自律的にタスクを完了するフレームワークFARを提案。失敗データを用いた選好学習と軽量な行動摂動により再試行を改善し、成功軌道を訓練に組み込むことで成功率を向上させる。
著者: Haoran Hao, Shahram Najam Syed, Jeffrey Ichnowski, Jeff Schneider
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
Robot policies inevitably encounter failures when deployed in real environments. Naive retries often repeat the same mistakes, while many existing recovery methods rely on human intervention. In this paper, we propose Failure-Aware Retry (FAR), a framework that enables robots to learn from previous failures at test time, adapt their behavior accordingly, and eventually complete the task autonomously. FAR combines Failure-Contrastive Preference Adaptation, which constructs preference learning data from failures to steer the policy away from previously unsuccessful behaviors, with lightweight action perturbations during retries to encourage local exploration. We further incorporate successful recovery trajectories into a training loop for continual policy improvement. Experiments in both simulation and real-world manipulation tasks show that FAR substantially improves success rates and robustness, with average gains of 17.6% over the standard diffusion policy in simulation and 11.7% in the real world. In addition, FAR significantly improves data efficiency under both reset and timestep budgets during continual policy improvement by exploiting informative failure cases.