Nvidia boosts robotics with Project GR00T
Nvidia has announced Project GR00T, a general-purpose foundation model for humanoid robots which is designed to further its work driving breakthroughs in robotics and embodied AI.
As part of the initiative, the company also unveiled a new computer, Jetson Thor, for humanoid robots based on the Nvidia Thor system-on-a-chip (SoC), as well as significant upgrades to the Nvidia Isaac robotics platform including generative AI (GenAI) foundation models and tools for simulation and AI workflow infrastructure.
“Building foundation models for general humanoid robots is one of the most exciting problems to solve in AI today,” says Jensen Huang, founder and CEO of Nvidia. “The enabling technologies are coming together for leading roboticists around the world to take giant leaps towards artificial general robotics.”
Robots powered by GR00T, an acronym for Generalist Robot 00 Technology, will be designed to understand natural language and emulate movements by observing human actions – quickly learning coordination, dexterity, and other skills in order to navigate, adapt, and interact with the real world.
In his GTC keynote, Huang demonstrated several such robots completing a variety of tasks.
Built for humanoid robots
Jetson Thor was created as a new computing platform capable of performing complex tasks and interacting safely and naturally with people and machines. It has a modular architecture optimised for performance, power, and size.
The SoC includes a next-generation GPU based on the Nvidia Blackwell architecture with a transformer engine delivering 800 teraflops of 8-bit floating point AI performance to run multimodal GenAI models like GR00T. With an integrated functional safety processor, a high-performance CPU cluster, and 100GB of Ethernet bandwidth, it significantly simplifies design and integration efforts.
Nvidia is building a comprehensive AI platform for leading humanoid robot companies such as 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, Unitree Robotics, and XPENG Robotics, among others.
“We are at an inflection point in history, with human-centric robots like Digit poised to change labour forever,” says Jonathan Hurst, co-founder and chief robot officer at Agility Robotics. “Modern AI will accelerate development, paving the way for robots like Digit to help people in all aspects of daily life. We’re excited to partner with Nvidia to invest in the computing, simulation tools, machine learning environments, and other necessary infrastructure to enable the dream of robots being a part of daily life.”
Geordie Rose, co-founder and CEO of Sanctuary AI, adds: “Embodied AI will not only help address some of humanity’s biggest challenges, but also create innovations which are currently beyond our reach or imagination. “Technology this important shouldn’t be built in silos which is why we prioritise long-term partners like Nvidia.”
Major updates to Isaac platform
The Isaac tools that GR00T utilizes are capable of creating new foundation models for any robot embodiment in any environment. Among these tools are Isaac Lab for reinforcement learning and OSMO, a compute orchestration service.
Embodied AI models require massive amounts of real and synthetic data. The new Isaac Lab is a GPU-accelerated, lightweight, performance-optimised application built on Isaac Sim specifically for running thousands of parallel simulations for robot learning.
To scale robot development workloads across heterogeneous compute, OSMO coordinates the data generation, model training, and software/hardware-in-the-loop workflows across distributed environments.
The company also announced Isaac Manipulator and Isaac Perceptor – a collection of robotics pre-trained models, libraries, and reference hardware.
Isaac Manipulator offers state-of-the-art dexterity and modular AI capabilities for robotic arms with a robust collection of foundation models and GPU-accelerated libraries. It provides up to an 80x speed-up in path planning and zero-shot perception increases efficiency and throughput, enabling developers to automate a greater number of new robotic tasks. Among early ecosystem partners are Yaskawa, Universal Robots, a Teradyne company, PickNik Robotics, Solomon, READY Robotics, and Franka Robotics.
Isaac Perceptor provides multi-camera, 3D surround-vision capabilities which are increasingly being used in autonomous mobile robots adopted in manufacturing and fulfillment operations to improve efficiency and worker safety, as well as reduce error rates and costs. Early adopters include ArcBest, BYD, and KION Group as they aim to achieve new levels of autonomy in material handling operations and more.
