NVIDIA and Researchers Introduce RoboTTT, Extending Robot Policy Context to 8K Time Steps

Researchers from NVIDIA, Stanford University and The University of Texas at Austin published RoboTTT, a robot policy that extends visual and motor context to 8K time steps, along with its training method. The team said it expands context length to roughly three orders of magnitude beyond previous state-of-the-art policies without increasing inference latency. On real-robot manipulation tasks, it improved overall performance by 87% over a single-step-context baseline and completed a 10-step assembly task that lasted five minutes, the team said.

The team says RoboTTT is designed to enable imitation using a human video as a single context and to handle long assembly tasks. By storing history in fast weights and updating them during inference, it takes a different approach to incorporating past information than policies built around short visual and motor contexts.

The technical focus is not the 8K context length itself, but the design for handling long histories without increasing inference latency. Rather than retaining the history as is, the team compresses it into fast weights that are also updated during inference. RoboTTT therefore extends not simply the input length, but the way past observations and actions are continually incorporated into a policy's internal state. However, the claim that inference latency does not increase comes from the research team; the materials provided do not specify measured latency on real robots or the breakdown of computing resources. The results also need to be distinguished. The team reported a 62% difference in a comparison between 8K and 1K context, and an 87% improvement over a single-step-context baseline. Because the comparators differ, these should not be treated as successive gains on the same measure. Completing a 10-step assembly task lasting five minutes is an example of applying the approach to long, multistep manipulation, but the evaluation described in the materials concerns specific real-robot manipulation tasks and does not demonstrate generalization to different tasks or environments. Related coverage by this newspaper includes Nikkei XTECH reports on humanoid control technology using world models, which examines reinforcement learning with world models and sim-to-real transfer for humanoids. NVIDIA introduces GR00T 2 and Isaac Lab 3.0 at CoRL also offers a point of comparison for the technological stages of robot learning. Each involves different technologies, while RoboTTT focuses on how a policy takes in a long history. It is important to distinguish the stages at which performance is achieved: the model, simulation or real-robot control. The industry implication is that the research presents a learning and adaptation pathway for using long manipulation histories and human demonstrations while a policy is running. The team also describes one-shot in-context imitation using human videos and a method for learning from human corrections to failed actions used as context. The materials do not say how much additional data or human input these methods require in practice, or how online updates to fast weights affect safety and reproducibility. The next points to examine are success rates and trial counts by task, measured latency and computing requirements, applicability to different robots and tasks, and the conditions for video and correction data.

RoboTTT is a model that incorporates Test-Time Training (TTT) into a vision-language-action (VLA) policy and was implemented on the basis of GR00T N1.7. Instead of caching history, it compresses and stores it in "fast weights" updated through gradient descent. The team says this mechanism uses a fixed-size state to condition on a long history and updates the state during both training and inference. Training uses truncated backpropagation through time (TBPTT), which divides sequences and limits the intervals through which gradients are propagated, while retaining fast weights across intervals. The team said a model trained with an 8K time-step context outperformed the same model pretrained with 1K context by 62%. It also reported one-shot in-context imitation using human videos, robustness to disturbances and improved performance on long, multistep tasks.