タスク特化型視覚言語制御のための弱教師あり潜在モデル
Weakly-supervised Latent Models for Task-specific Visual-Language Control
目標状態の教師信号のみから共有潜在空間における行動による状態変化を学習するタスク特化型潜在ダイナミクスモデルを提案し、ドローンの空間的グラウンディングタスクで成功率を58%から71%に改善した。
著者: Xian Yeow Lee, Lasitha Vidyaratne, Gregory Sin, Ahmed Farahat, Chetan Gupta
分類: cs.AI, cs.LG, eess.SY
原文アブストラクト
Autonomous inspection in hazardous environments requires AI agents that can interpret high-level goals and execute precise control. A key capability for such agents is spatial grounding, for example when a drone must center a detected object in its camera view to enable reliable inspection. While large language models provide a natural interface for specifying goals, using them directly for visual control achieves only 58\% success in this task. We envision that equipping agents with a world model as a tool would allow them to roll out candidate actions and perform better in spatially grounded settings, but conventional world models are data and compute intensive. To address this, we propose a task-specific latent dynamics model that learns state-specific action-induced shifts in a shared latent space using only goal-state supervision. The model leverages global action embeddings and complementary training losses to stabilize learning. In experiments, our approach achieves 71\% success and generalizes to unseen images and instructions, highlighting the potential of compact, domain-specific latent dynamics models for spatial alignment in autonomous inspection.