介護施設向けサービスロボットのマルチモーダル言語モデル駆動による対話と伴走
Multimodal-Language-Model-Driven Interaction and Companionship for Service Robots in Elderly-Care Facilities
介護施設のサービスロボットに、能動的な人物追従、LLMによる音声対話、VLMによる転倒検知を統合したシステムを提案し、実験でその有効性を示した。
著者: Ching-Chieh Liu, Cong-Thanh Vu, Yen-Chen Liu
分類: cs.RO, eess.SY
原文アブストラクト
Service robots are increasingly deployed in elderly-care facilities to alleviate caregiver workload and enhance the quality of daily care. However, most existing studies focus on isolated service functions and lack integrated capabilities for continuous companionship, natural interaction, and safety monitoring. In this paper, we present an intelligent companion robot system that unifies active visual human-following, real-time LLM-driven speech interaction for intent understanding and task execution, and VLM-based safety monitoring for fall detection and abnormal posture assessment. The perception layer ensures robust human tracking and uses an active gimbal to maintain the user in view during occlusions or abrupt movements. At the interaction layer, a Large Language Model interprets spoken requests and maps them to robot actions, enabling escorting and semantic navigation. Simultaneously, a VLM-based safety agent continuously analyzes visual observations to detect fall-related or abnormal postures and triggers emergency responses when necessary. Experimental results demonstrate the system's ability to reliably follow and interact with humans, while effectively detecting potential falls to ensure user safety.