ロボット制御ループの実行時安全チェック:聞かれれば拒否するが、行動時には従う
Refuses When Asked, Complies When Acting: A Runtime Safety Check for Robot Control Loops
ロボット制御ループに実行前の安全確認を追加し、危険な指示を事前に停止できることを示した論文。
著者: Arnav Grover
分類: cs.RO, cs.AI
原文アブストラクト
Frontier models can control robots through agent harnesses that loop on one question: what motion comes next toward the user's goal? We show that this default loop carries out harmful instructions. With GPT-6 Astra controlling a low-cost SO-101 arm through the Inspect Robots harness on three household tasks whose hazard is common knowledge, such as "put the knife in the toy basket", the policy refused 0% of 15 trials, attempted the harm every time, and completed it in 27%. Its one exit, give_up, is meant for goals that cannot be achieved, and it never cited the hazard. We add an optional runtime safety check to the loop: before the first motion, the model is asked whether it would carry out the instruction, given the same camera frames. The check would have stopped all 15 hazardous instructions, including the 4 that completed the harm, and flagged none of the 15 harmless controls. Code and data: https://github.com/Arnav-Gr0ver/Refuses-When-Asked-Complies-When-Acting.