PACT: 拡散ポリシーの身体的操作における自己進化型物理安全性アライメント
PACT: Self-Evolving Physical Safety Alignment for Diffusion Policies in Embodied Manipulation
事前学習済みの拡散ポリシーを、デモデータや報酬なしで制約適合領域に投影する自己進化型の後処理フレームワークを提案し、安全性違反を平均31.0%削減しつつタスク成功率を30.7%向上させた。
著者: Lingxuan Wu, Zijian Zhu, Lizhong Wang, Chengyang Ying, Huayu Chen, Xiao Yang, Fangming Liu, Jun Zhu
分類: cs.RO, cs.AI
原文アブストラクト
Diffusion policies have achieved remarkable success in robotic manipulation, yet they often fail to satisfy strict physical constraints required for safe deployment. Existing approaches impose safety either prematurely during training or reactively via external guardrails at test time, limiting policy expressivity and overall scalability. We propose Physical safety Alignment for Constrained Trajectories (PACT), a self-evolving post-training framework that projects pretrained diffusion policies onto constraint-feasible regions without accessing demonstration data or task rewards. PACT distills constraint gradients into the diffusion model through a reverse-KL objective with dense supervision across timesteps. It incorporates a curriculum that progressively tightens constraints while maintaining theoretically bounded policy shift and monotone improvement, mitigating the safety-performance trade-off from catastrophic forgetting. On simulated and real-world embodied manipulation benchmarks, PACT significantly reduces safety violations by 31.0% on average while improving task success by 30.7%.