Editor’s notice: This submit is a part of Into the Omniverse, a collection centered on how builders, 3D practitioners and enterprises can rework their workflows utilizing the most recent developments in OpenUSD and NVIDIA Omniverse.
Open supply has grow to be important for driving innovation in robotics and autonomy. By offering entry to essential infrastructure — from simulation frameworks to AI fashions — NVIDIA is enabling collaborative growth that accelerates the trail to safer, extra succesful autonomous methods.
At CES earlier this month, NVIDIA launched a brand new suite of open bodily AI fashions and frameworks to speed up the event of humanoids, autonomous autos and different bodily AI embodiments. These instruments span your complete robotics growth lifecycle — from high-fidelity world simulation and artificial knowledge technology to cloud-native orchestration and edge deployment — giving builders a modular toolkit to construct autonomous methods that may purpose, be taught and act in the true world.
OpenUSD supplies the frequent framework that standardizes how 3D knowledge is shared throughout these bodily AI instruments, enabling builders to construct correct digital twins and reuse them seamlessly from simulation to deployment. NVIDIA Omniverse libraries, constructed on OpenUSD, function the supply of floor‑fact simulation that feeds your complete stack.
From Labs to the Present Ground
At CES 2026, builders introduced the NVIDIA bodily AI stack out of the lab and onto the present ground, debuting machines starting from heavy tools and manufacturing facility assistants to social and repair robots.
The stack faucets into NVIDIA Cosmos world fashions; NVIDIA Isaac applied sciences, together with the brand new Isaac Lab-Enviornment open supply framework for coverage analysis; the NVIDIA Alpamayo open portfolio of AI fashions, simulation frameworks and bodily AI datasets for autonomous autos; and the NVIDIA OSMO framework to orchestrate coaching throughout compute environments.
Caterpillar’s Cat AI Assistant, powered by NVIDIA Nemotron open fashions for agentic AI and working on the NVIDIA Jetson Thor edge AI module, brings pure language interplay instantly into the cab of heavy autos. Operators can ask “Hey Cat”-style questions and get step‑by‑step steering, in addition to modify security parameters by voice.
Behind the scenes, Caterpillar makes use of Omniverse libraries to construct manufacturing facility and job‑website digital twins that may assist simulate layouts, site visitors patterns and multi‑machine workflows. These insights are fed again into tools and fleets earlier than adjustments are deployed to job websites, making AI‑assisted operations safer and extra environment friendly.
LEM Surgical showcased its Dynamis Robotic Surgical System, which is FDA-cleared and in routine medical use for spinal procedures. The subsequent-generation system makes use of NVIDIA Jetson AGX Thor for compute, NVIDIA Holoscan for real-time sensor processing and NVIDIA Isaac for Healthcare to coach its autonomous arms.
LEM Surgical additionally makes use of NVIDIA Cosmos Switch — an open, totally customizable world mannequin that permits bodily primarily based artificial knowledge technology — to generate artificial coaching knowledge and the NVIDIA Isaac Sim framework for digital twin simulation. Designed as a dual-arm humanoid surgical robotic for hard-tissue surgical procedure, the Dynamis system mimics human surgeon dexterity and allows complicated spinal procedures with enhanced precision, assuaging strenuous bodily calls for on surgeons and surgical assistants.
NEURA Robotics is constructing cognitive robots on a full NVIDIA stack, utilizing Isaac Sim and Isaac Lab to coach its 4NE1 humanoid and MiPA service robots in OpenUSD‑primarily based digital twins earlier than deployment in home settings and workplaces. The corporate used NVIDIA Isaac GR00T‑Mimic to submit‑prepare the Isaac GR00T basis mannequin for its platforms.
As well as, NEURA Robotics is collaborating with SAP and NVIDIA to combine SAP’s Joule brokers with its robots, utilizing the Mega NVIDIA Omniverse Blueprint to simulate and refine robotic conduct in complicated, lifelike operational situations earlier than these brokers and behaviors are deployed into the corporate’s Neuraverse ecosystem, in addition to in actual‑world fleets.
AgiBot makes use of NVIDIA Cosmos Predict 2 because the world‑modeling spine for its Genie Envisioner (GE-Sim) platform — permitting the platform to generate motion‑conditioned movies grounded in robust visible and bodily priors. Combining this knowledge with Isaac Sim and Isaac Lab, in addition to submit‑coaching on AgiBot’s personal knowledge, lets insurance policies developed in Genie Envisioner switch extra reliably to Genie2 humanoids and compact Jetson Thor-powered tabletop robots.
Intbot is utilizing the NVIDIA Cosmos Purpose 2 open mannequin to provide its social robots a “sixth sense” for the true world — utilizing the mannequin’s reasoning capabilities to determine easy social cues and security context that transcend easy scripted duties. In its Cosmos Cookbook recipe, Intbot demonstrates how reasoning imaginative and prescient language fashions can assist robots in deciding when to talk and how one can extra naturally work together with people.
How Robotics Builders Are Utilizing New Toolkits and Frameworks
NVIDIA lately launched Agile, an Isaac Lab-based engine for humanoid loco‑manipulation that packages a full, sim‑to‑actual‑verified workflow for coaching sturdy reinforcement studying insurance policies on platforms just like the Unitree G1 and LimX Dynamics TRON.
Robotics builders can use Agile’s constructed‑in job configurations, Markov Choice Course of mathematical fashions for decision-making, coaching utilities and deterministic analysis instruments to tune insurance policies. Builders can then stress‑check these insurance policies in Isaac Lab and switch locomotion and entire‑physique behaviors to real-world robots extra reliably and effectively.
Hugging Face and NVIDIA are bringing collectively their robotics communities by integrating NVIDIA Isaac GR00T N fashions and simulation frameworks into the LeRobot ecosystem. Builders can now entry Isaac GR00T N1.6 fashions and Isaac Lab‑Enviornment instantly inside LeRobot to streamline coverage coaching and analysis.
Plus, Hugging Face’s open‑supply Reachy 2 humanoid is now totally interoperable with NVIDIA Jetson Thor, enabling the direct deployment of superior imaginative and prescient language motion (VLA) fashions for sturdy actual‑world efficiency.
ROBOTIS, a number one developer of good servos, industrial actuators, manipulators, open-source humanoid platforms and academic robotic kits, constructed an open supply sim-to-real pipeline utilizing NVIDIA Isaac applied sciences. The workflow begins with excessive‑constancy knowledge technology in Isaac Sim, scales up coaching units utilizing GR00T‑Mimic for augmentation after which superb‑tunes a VLA‑primarily based Isaac GR00T N mannequin that deploys on to {hardware} — accelerating the transition from simulation to sturdy actual‑world duties.
Get Plugged In
Be taught extra about OpenUSD and robotics growth by exploring these sources:
- Learn this technical weblog to discover ways to develop generalist humanoid capabilities with NVIDIA Isaac and GR00T N1.6.
- Learn this technical weblog to discover ways to consider generalist robotic insurance policies in simulation utilizing NVIDIA Isaac Lab – Enviornment.
- Be taught how one can post-train Isaac GR00T with this two-part video tutorial.
- Watch NVIDIA founder and CEO Jensen Huang’s CES particular presentation.
- Enhance abilities for robotics growth with the self-paced robotics studying path.
- Take part within the Cosmos Cookoff, a hands-on bodily AI problem the place builders use Cosmos Purpose to energy robotics, autonomous methods and imaginative and prescient AI workflows.
