NVIDIA Research introduces VFlash for parallel generation of visual action plans

research.nvidia.com introduced VFlash, a decoding framework for physical AI. VFlash uses a lightweight generator to produce visual plans, such as points and trajectories, in parallel, primitive by primitive. According to the page, it reduced average 3D displacement error by 42.6% compared with an autoregressive method across three robot datasets and three target models, and recorded an average success rate of 70.2% in manipulation evaluations on MolmoSpaces.

In the MolmoSpaces evaluation, the success rate was 70.2% when visual plans were used, compared with 63.7% when image features were substituted. The comparison on the research page indicates that geometric information, such as coordinates and trajectories, could be important alongside language instructions as input to robot manipulation models.

The key point of the research is not that it accelerated robot inference as a whole, but that it replaced sequential coordinate-token generation with parallel generation by primitive at the stage that outputs visual plans. Because coordinate points and trajectories have known structure, the generation units are aligned with groupings such as points, boxes and trajectories rather than fixed-length token windows. The output length is specified through tool calls, and the type of primitive determines the coordinate arrangement. In other words, the speedup comes not from adding computing resources but from incorporating the structure of the output into the decoding method. The results need to be read separately by use case. For robot trajectories, the method reduced average 3D displacement error by 42.6% across three datasets and three target models; on MolmoSpaces, it achieved a 70.2% success rate. By contrast, box generation on COCO and Dense200 was measured using a separate metric: 2.6 to 2.8 times as many boxes per second. These involve different tasks and evaluation metrics, and cannot be combined into a single figure and interpreted as an overall improvement in robot performance. On MolmoSpaces, the success rate was 63.7% when trajectory information from VFlash was removed and replaced with image features, compared with 70.2% using 2D and 3D plans. The comparison on the page suggests that generated geometric information may be relevant to downstream manipulation, but it does not establish general performance across different operations and environments. VFlash is not a standalone controller that handles everything from perception to action. A fixed VLM produces plan features, which a downstream action model references to generate actions. On MolmoSpaces, a model based on Cosmos-Predict2.5 and additionally trained for control predicts 20-step action chunks and updates the plan every three chunks. This division of labor shows that whether faster generation translates into successful manipulation should be evaluated in conjunction with the downstream model configuration. VFlash's speed metrics alone do not reveal the response time or operating conditions of the complete system. The next questions are whether the reported speed, error and success-rate results hold in different environments and real-world conditions, and how errors in generated plans affect safety. The introduction page presents evaluation results and the model configuration, but does not discuss operational uptime, a breakdown of latency, the scale of the training data or safety mechanisms for failures. For now, the research is best viewed as demonstrating the effectiveness of parallel generation on evaluation benchmarks; its applicability to complete control systems will require separate verification.

VFlash trains a lightweight generator on annotated geometric data while keeping the target vision-language model (VLM) fixed. The generator is a five-layer noncausal transformer that predicts variable-length primitives in a single pass. It uses dedicated coordinate tokens or fixed-width numeric fields to reduce coordinate misalignment. On MolmoSpaces, single-rollout generation with an H100 and batch size 1 is reported to be about 25 times faster. On COCO and Dense200, box generation throughput was 2.6 to 2.8 times higher, while average F1 was close to that of the autoregressive method. The robot manipulation evaluation used a world action model based on Cosmos-Predict2.5 and additionally trained for control. The model predicts 20-step action chunks and updates the plan every three chunks. Average success rates on MolmoSpaces were 62.6% for the baseline, 61.0% for autoregressive planning, 69.3% for VFlash with 3D plans only, and 70.2% for VFlash with 2D and 3D plans. The page says the results were compared using the same action-model configuration.