Instant-Fold: 変形物操作のための文脈内模倣学習
Instant-Fold: In-Context Imitation Learning for Deformable Object Manipulation
単一の人間デモから変形物の多様な操作モードを推論・実行する文脈内模倣学習フレームワークを提案し、シミュレーションのみで訓練し実世界へゼロショット転移する。
著者: Yilong Wang, Cheng Qian, Edward Johns
分類: cs.RO, cs.AI, cs.CV
原文アブストラクト
Deformable object manipulation (DOM) is challenging due to high-dimensional, partially observable states that evolve through long-horizon, topology-changing interactions with multiple valid manipulation modes. We introduce Instant-Fold, an in-context imitation learning framework for DOM. Given a single human demonstration, our policy infers and executes diverse manipulation modes directly from the demonstration, including variations in spatial execution and ordering, without requiring gradient updates. Our approach first learns deformation-aware visual representations via temporal contrastive pretraining, after which a flow-matching transformer policy conditioned on the demonstration predicts actions to execute the intended manipulation mode. Trained entirely in simulation, Instant-Fold generalizes across diverse folding modes and transfers zero-shot to real-world settings without additional data collection or finetuning. Videos are available at https://instant-fold.github.io.