HumanCOIN:一人称操作における人間の意図を記録する12,500時間データセット
HumanCOIN: A 12,500-Hour Dataset for Human Intent in Egocentric Manipulation
行動の背後にある意図(視線・筋電・脳波)を大規模に記録した一人称操作データセットHumanCOINを構築し、12,500時間分のマルチモーダルデータを公開した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Shihan Wu, You Liu, Chunyu He, Minghui Qin, Longhan Jiao, Jinbang Guo, Shaoxuan Xie, Kaixi Tian, Wenjie Chen, Jingrui Pang, Yuanhang Zhang, Yichi Zhang, Miao Liu, Tengfei Pan, Yance Jiao, Yunyan Zhang, Yihang Li, Yongjian Shen, Li Liu, Kai Zhu, Chen Fang, Jianxing Chen, Sihan Liu, Xiaofeng Wang, Jie Zhao, Jing Li, Yueqi Wang, Yichen Tang, Wei Li, Zhexuan Yang, Yue Cao, Ji Zhao, Kai Wang, Dongsheng Jiang, Yongwei Wang, Wei Mu, Jiangmiao Pang, Maoqing Yao, Wenchao Ding, Zheng Zhu, Zhongpu Xia, Dalong Du, Jianwei Niu, Dahui Gao, Zhiwei Yu, Bo Lei, Huan Yang, Yonghua Lin, Tiejun Huang, Zhongyuan Wang, Guocai Yao
分類: cs.CV, cs.RO
原文アブストラクト
First-person manipulation datasets treat the observation-action pair as the atomic unit and compress the generation and execution of an action into a policy network's weights, rendering the intent state behind the action unobservable. Human motor control research shows that intent mediates between observation and action and leaves observable traces along the intent pathway from decision to effect: gaze reaches the target before the hand, muscle load builds before contact, and cortical activity precedes the motor command. To record this pathway at scale, we present HumanCOIN, a first-person manipulation dataset built around human intent: 12,500 hours across 38 scene classes with three tiers of proxies. At the output end, the L1 attention tier (10,000 h) infers gaze from headband camera video, exposing the operator's next object of attention and reach before the hand moves. Midway, the L2 exertion tier (2,000 h) records surface EMG from armbands, making load expectations readable before execution is complete. At the origin, the L3 cognition tier (500 h) records scalp EEG, giving early readouts of task decisions and monitoring. Each tier inherits all proxies from the tiers below. The dataset further provides multi-view RGB images, depth maps and point clouds, 6-DoF head and wrist pose, and step-level action and object annotations. The controlled L3 setting additionally measures gaze, hand pose and finger kinematics directly. Clock and calibration data are released with the dataset so that any read rate can be re-aligned. The three tiers cover three stations, from visual orienting to cortical preparation, making the process behind visual-action pairs available for modeling at dataset scale. Project page: https://flagopen.github.io/HumanCOIN.
