物体中心再構成による視覚ベース3次元力推定
Object-Centered Reconstruction for Vision-Based 3D Force Estimation
ステレオ内視鏡映像から軟組織の変形を解析し、物体中心座標系で点群を再構成してニューラルネットワークで3次元力を推定する手法を提案。ファントムと豚結腸で精度を検証した。
著者: Zhonghao Zhang, Mingyeung Wu, Hao Yang, Ayberk Acar, Alan Kuntz, Jie Ying Wu
分類: cs.RO
原文アブストラクト
Excessive force may damage tissue and increase the risk of anastomotic leakage in robotic colorectal surgery. Although the da Vinci 5 provides force sensing, this capability is unavailable on earlier da Vinci systems and many other surgical robotic platforms. In this work, we present a vision-based pipeline for estimating 3D interaction forces from soft-tissue deformation in stereo endoscopic video. We dynamically reconstruct the tissue point cloud in an object-centered coordinate frame, track tissue points with geometric constraints, and predict the 3D force vector with a neural network. We progressively evaluate the pipeline on rubber-glove phantoms, ex vivo porcine colons, and in vivo colorectal surgical video sequences. Under varying tissue orientations and positions within the endoscopic view, as well as different camera viewpoints, the proposed method achieves average root mean square error (RMSEs) of 0.77 N and 1.30 N on the phantom and porcine colon, respectively. Compared with the camera-frame representation, the object-centered representation reduces average RMSE by 51.3% and 56.7%, while geometry-constrained tracking reduces RMSE by 19.8% and 25.3% compared with CoTracker. We further qualitatively demonstrate the feasibility of vision-based force estimation on an in vivo colorectal surgical sequence, as a step toward clinical translation of vision-based, sensorless force estimation.