Technical Directions
Robotics coverage relating to world models.
247 selected reportsLast 30 days 203 recordsAll coverage 1,072 records
NVIDIA introduces Cosmos-H-Dreams, a real-time, action-conditioned generative simulator for surgical robotics. It distills the capabilities of Cosmos-H-Surgical-Simulator into a causal, few-step student model, enabling interactive environments for policy evaluation and synthetic data generation. The system runs on a single NVIDIA RTX PRO 6000 GPU and supports integration with surgical platforms like Versius.
NVIDIA has released an open-source, GPU-accelerated Medical Physics Simulation framework as part of its Isaac for Healthcare platform. This tool enables developers to model anatomy-device interactions, generate complex scenarios, and train robot policies in simulation, significantly reducing development time and improving regulatory readiness.
NVIDIA and Hugging Face are releasing the NVIDIA Isaac GR00T 1.7 model and Isaac Teleop framework into LeRobot, an open-source robotics library, to provide developers with shared tools for training, evaluating, and deploying robot foundation models. NVIDIA Cosmos 3, a frontier world model for physical AI, is also set to be integrated soon.
LeRobot v0.6.0 introduces world model policies that learn to imagine the future, new VLAs, and a reward models API. It also includes six new simulation benchmarks, improved dataset handling with depth support and language annotations, and deployment tools like lerobot-rollout.
Dexterity introduces Foresight, a world model that enables robots to reason about the physical world, predict outcomes, and act confidently in real environments. Trained on over 100 million autonomous actions in production, Foresight is designed for interpretability, safety, and performance, with capabilities like predictive branching and pragmatic decision-making. It addresses complex tasks like spatial packing and includes a game and API challenge for further exploration.
Dexterity introduces 'Transactable World Models' as a foundational component for Physical AI, emphasizing interpretability, physics integration, and real-time consistency. The approach treats world models as operators rather than data stores, enabling multi-agent coordination and robust manipulation in complex environments. This contrasts with traditional world models that focus on data compression and replay, lacking causal reasoning and physical grounding.
Hugging Face announces LeRobot v0.4.0, a major upgrade for open-source robotics with Dataset v3.0, new VLA models like PI0.5 and GR00T N1.5, and a plugin system for hardware integration. The release also adds support for LIBERO and Meta-World simulations, multi-GPU training, and a new Hugging Face Robot Learning Course.
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).