触覚生成のための潜在空間選択：画像潜在表現は素材認識を保持するか
Where to Generate Touch: Do Image Latents Preserve Material Recognition?
事前学習済み画像モデルの潜在空間で触覚を再構成し、素材分類器の性能劣化を比較してSD-VAE潜在空間が触覚生成に適することを示した論文。
著者: Wanghao Ye, Aarosh Das, Sihan Chen, Yiting Wang, Bowei Tian, Guoheng Sun, Shwai He, Zheyu Shen, Ziyao Wang, Yexiao He, Zhaoyi Liu, Meng Liu, Yuning Zhang, Meng Feng, Ziyi Wang, Yilong Dai, Yifei Dong, Siyuan Peng, Zhenle Duan, Joshua Liu, Lang Xiong, Daeun Jung, Aditya Sarkar, Parastoo Pilevar, Xiaochuan Gong, Yancheng Zhu, Yingying Fan, Qiong Wang, Ang Li
分類: cs.CV, cs.RO
原文アブストラクト
Generated touch could extend robot learning beyond recorded contacts. A tactile generator needs a latent space, and we choose one by testing whether a material classifier trained on real GelSight presses still works on encode-decode reconstructions from pretrained image models. On ObjectFolder Real, the classifier loses 6.5 to 9.7 points on reconstructions from three 16x discrete tokenizers and 0.3 points on those from the Stable Diffusion VAE (SD-VAE). The 16x tokenizers again lose far more on Touch and Go and on DIGIT object recognition (YCB-Slide). Retraining the classifier on reconstructions, with no new labels, narrows the discrete gap. SD-VAE reconstructions also keep object identity, with 76.7% object-matching accuracy against 78.9% for real touch and 67.0% for MAGVIT-v2. We therefore select SD-VAE latents and show a photo-to-touch generator trained in them. Code will be released upon acceptance.