記憶拡張リフレクションによるエージェント型人間動作インタラクション
InteractAgent: Agentic Human Motion Interaction with Memory-Augmented Reflection
MLLMをプランナ兼評価者、凍結MotionLLMを実行者として、計画・実行・観察・反省を閉ループで回し、長期的な人間-シーンインタラクションを実現するエージェントシステム。
著者: Tao Zhang, Senhe Zhang, Zeyu Zhang, Yujia Zhang, Dong Gong
分類: cs.RO, cs.CV
原文アブストラクト
Long-horizon human-scene interaction requires coordinated spatial reasoning, physical feasibility, and the ability to recover from compounding execution errors. Text-driven motion models are largely scene-agnostic and lack closed-loop correction, while geometry-conditioned approaches depend on paired scene-motion training or explicit 3D preprocessing that limits generalization to novel layouts and open-vocabulary goals. We present InteractAgent, an agentic multimodal LLM system for long-horizon human-scene interaction that plans, executes, observes, and reflects in a closed loop, using a single MLLM as both planner and evaluator and a frozen MotionLLM as the executor. InteractAgent integrates Online Observation for step-wise recovery decisions (continue/redo/replan) and Memory-Augmented Reflection for multi-round plan refinement with adaptive stopping. On UniHSI, InteractAgent improves task success and reduces penetration compared with training-free baselines. Code: https://github.com/AIGeeksGroup/InteractAgent.