Apptronik to integrate NVIDIA robotics foundation model into humanoid Apollo

On March 18, 2024, Apptronik announced a collaboration with NVIDIA to enhance learning for humanoid robots. The companies plan to integrate NVIDIA’s robotics foundation model Project GR00T into Apptronik’s Apollo, with the aim of treating text, video and human demonstrations as task instructions. At NVIDIA’s GTC keynote, a demo showed Apollo operating a juicer and serving juice to a person.

Apptronik’s announcement outlined a system in which Apollo receives text, video and human demonstrations as task instructions and generates actions, and included a demo of the robot operating a juicer. The company said Apollo can perceive its environment and predict its next action, rather than simply repeat previously performed movements. Whether it can demonstrate reproducibility and safety in real-world tasks will factor into adoption decisions.

The core of the collaboration is not a redesign of the humanoid or a production plan, but the software and computing setup for learning and executing tasks. Apptronik said it plans to integrate Apollo with NVIDIA’s Project GR00T and described a process in which the robot uses text, video and human demonstrations to learn skills and translate them into actions. The juicer demo at GTC illustrated that setup with a specific task, but the demonstration alone does not establish its applicability to other tasks or its reliability in real-world operations. The new elements are the planned integration with GR00T and the use of NVIDIA’s robotics technologies, including Isaac Lab and OSMO. Apollo, however, was announced in August 2023. Apptronik has also described its use of Jetson AGX Orin and Jetson Orin NX as its main computing systems, along with linear actuators, replaceable batteries and a modular design with multiple mounting configurations. Thus, the announcement concerns plans to combine an existing robot with learning models and development and computing platforms, not the start of mass production. It does not specify annual production volumes or deployment scale. The architecture pairs NVIDIA’s models, development environment and computing resources with Apollo’s hardware, linking task instructions such as demonstrations to the robot’s actions. Because the announcement names Jetson products as the robot’s onboard computers, a key focus is how the models, development and computing platforms work with the physical robot, rather than evaluating each separately. However, the announcement does not specify the amount of data or training time required, task-specific success rates or how models will be updated. Without that information, it is not possible to assess whether the skills shown in the demo can be reproduced in different environments or for other tasks. Our related article, 36Kr reports OpenAI’s robotics team disbanded; Fortune confirms restructuring, covered reports about a reorganization of a robotics team. This announcement concerns a technical collaboration between a model and a robot, not a corporate restructuring, and there is no basis for linking the two as part of the same development. The points to watch are whether Apollo can reproduce skills, expand the range of tasks it can perform, demonstrate safety through force control, and clarify the division of responsibilities between development platforms and real-world operation. The announced demo and specifications do not establish these operational conditions.

According to Apptronik, Apollo uses NVIDIA Jetson AGX Orin and Jetson Orin NX onboard as its main computing systems. The collaboration will also use GR00T, Isaac Lab and OSMO, a computing orchestration service. The announcement outlined plans for the robot to learn skills such as coordinated movement and dexterity from video and human demonstrations, then generate actions on the robot. Apptronik says Apollo uses linear rather than rotary actuators and has a replaceable battery that provides 4 hours of operation. Its modular design allows the upper body to be mounted on legs, a wheeled base or a pedestal. The company has also described a force-control system for moving Apollo’s arms near people.