コード飛行:能動的な視覚・物理フィードバックによる空中マニピュレーションのための身体性コーディングエージェント
Fly-by-Code: Embodied Coding Agents for Aerial Manipulation with Active Visual and Physical Feedback
空中マニピュレーションでは視野制限や動的結合により実行フィードバックが不十分になる問題に対し、未観測の結果を確認する「View」と失敗仮説を検証する「Probe」からなる能動フィードバックを導入し、タスク成功率を31.7%から75%に改善した。
著者: Jaewoo Lee, Jeongyeon Seo, Sihyun Cho, Gyeongrak Choe, Yutong Wang, Bavin Saravanan, Jia-Bin Huang, Furong Huang, Sebastian Scherer, Guanya Shi, H. Jin Kim, Seungjae Lee, Dongjae Lee
分類: cs.RO, cs.CV, cs.LG
原文アブストラクト
Embodied coding agents construct robot policies from natural-language instructions by generating executable programs and refining them from execution feedback. This refinement assumes that execution feedback reveals whether the task succeeded and, after a failure, provides sufficient evidence to guide program revision. Aerial manipulation violates this assumption in two ways. Payload constraints restrict onboard camera coverage and leave task-critical outcomes outside the robot's view (partial observability), while strong dynamic coupling and limited control margins make physical interactions fragile (limited interaction robustness) and leave failure causes unresolved. Re-examining the same feedback cannot recover observations that were never collected or separate failure causes that the available evidence does not distinguish. To acquire this missing evidence, we introduce Active Feedback (AF), an active "View and Probe" framework. When execution feedback leaves unresolved questions, the agent synthesizes a separate diagnostic program that runs in simulation to inspect unseen outcomes from informative viewpoints (View) or test hypotheses about failure causes through targeted physical interactions (Probe). Across six simulated aerial manipulation tasks with 60 trials per condition, AF reduces false completion declarations from 25 to 5 and raises final task success from 31.7% to 75% compared with terminal feedback under the same coding-agent configuration and task-execution limit. Using shared robot APIs, task programs refined with AF in simulation run on a physical aerial manipulator without task-program source-code modification and complete multi-stage manipulation tasks in the real world. These results show that executable diagnosis fills gaps in execution feedback for embodied coding agents, improving both completion assessment and program refinement in aerial manipulation.
関連論文
- HexaGripper: 単一アクチュエータ・ウインチ展開型グリッパによる自律空中荷物回収空中マニピュレーション
- 部分的な視覚観測による全身空中把持と持ち上げ空中マニピュレーション
- 局所全身VLA行動からシーン規模の空中マニピュレーションへ空中マニピュレーション
- 可動コンプライアントアンカーによるケーブル吊り下げ空中マニピュレーションの受動剛性成形空中マニピュレーション
- 空中グリッパー:勾配ベースのリアルタイム逆ゲーム予測・計画フレームワーク空中マニピュレーション
- QuadHand:MRC-SDFを用いた全身運動計画によるコンパクトなクアッドロータ空中マニピュレータ空中マニピュレーション