ImageWAM Uses Image-Editing Model’s Internal Representations to Generate Robot Actions

ImageWAM asked whether video generation is necessary for robot action generation and described a method that uses a pretrained image-editing model as the backbone of a robot policy. At inference time, it does not decode the edited image. Instead, an action expert uses the KV cache obtained from a single forward pass of the image-editing model to generate future robot actions.

In contrast to an architecture that generates video before predicting actions, ImageWAM passes the editing model’s KV cache to action generation and avoids decoding the edited image. This offers an alternative for separating visual representation from action generation when latency from video generation or visual information unnecessary for action is a concern, but the information presented does not establish the extent of any benefit.

ImageWAM’s design focuses not on rendering the world’s future state as video, but on passing task-relevant visual information to action generation. The description characterizes approaches that use video generation as facing several challenges: future tokens across multiple frames increase latency, use capacity for appearance information unrelated to the action, and, over longer prediction horizons, visual errors may lead to incorrect action predictions. In response, ImageWAM uses the KV cache obtained from the image-editing process, rather than the edited image itself. This choice can be read as shifting the role of a visual world model from “reproducing future video” to “representing changes in accordance with instructions.” It connects the idea that manipulation instructions specify what should change in a scene with an image-editing model’s ability to follow instructions, addressing task-relevant changes while preserving the current observation. The use of a separate action expert for action generation is also a design feature, dividing responsibilities between organizing visual information on the image-editing side and producing motion downstream. However, the information presented does not make it possible to determine how much visual information the KV cache retains or how it represents the temporal information required for action generation. The industry implication is that, when comparing robot foundation model designs, the issue is not only whether video is generated, but also how a visual model’s internal representations are connected to a controller. If an architecture that does not explicitly generate video proves viable, it could reduce computational load or latency, but that is the method’s aim and should be distinguished from demonstrated performance in measurements. A comparison would require latency, action success rates, the effect of errors over longer time horizons, and generalization to different objects and instructions under the same task conditions. The description provided contains no such figures, so it is not possible to conclude that the method is more efficient or more accurate than video-generation approaches. A key question going forward is whether the image-editing model’s ability to capture changes can be evaluated separately from the action expert’s ability to convert them into executable actions.