Technical Directions
Robotics coverage relating to sim-to-real.
145 selected reportsLast 30 days 114 recordsAll coverage 553 records
FedEx and Dexterity have expanded their collaboration to deploy Dexterity's Foresight world model and Mech trailer loading systems at the FedEx Hagerstown Hub. This marks a significant scaling of physical AI in logistics, enabling production at a larger operational scale with a focus on safety, consistency, and performance.
The US is banning the sale of foreign-made robots, including those from China, due to national security concerns. The FCC cites risks such as vulnerabilities in networked capabilities and potential exploitation of remote connectivity. The ban also extends to inverters with Wi-Fi, cellular, or Bluetooth control. Existing products already sold are unaffected, but companies can apply for approval to continue selling in the US.
LeRobot has converted the Hy-Embodied-0.5-VLA-RoboTwin model, achieving full forward parity with the original policy. The model supports 16D dual-arm pose/gripper actions and passes all validation checks, though it cannot handle more complex action spaces.
LeRobot has released a converted version of the Hy-Embodied-0.5-VLA-UMI model, ensuring full tensor compatibility and validation. The model supports a 16D dual-arm embodiment but cannot express 12D mobile-manipulator actions, indicating limitations in the current representation.
NVIDIA introduces Cosmos-H-Dreams, a real-time, action-conditioned generative simulator for surgical robotics. It distills the capabilities of Cosmos-H-Surgical-Simulator into a causal, few-step student model, enabling interactive environments for policy evaluation and synthetic data generation. The system runs on a single NVIDIA RTX PRO 6000 GPU and supports integration with surgical platforms like Versius.
NVIDIA has released an open-source, GPU-accelerated Medical Physics Simulation framework as part of its Isaac for Healthcare platform. This tool enables developers to model anatomy-device interactions, generate complex scenarios, and train robot policies in simulation, significantly reducing development time and improving regulatory readiness.
MIT's JARVIS Challenge explored whether AI can transform complex engineering tasks like building jet engines. Students used AI tools to design, fabricate, and test small gas turbine engines, but found that human expertise and judgment remained critical. The challenge revealed that while AI can speed up design and testing, physical manufacturing and engineering experience are still essential. The study emphasizes the need for a balance between AI assistance and human oversight in engineering workflows.
MIT researchers have developed SceneSmith, a system that uses AI agents and vision-language models to generate highly realistic and detailed virtual environments for robot training. These environments allow robots to practice tasks in simulated settings, reducing the need for physical testing. The system can create complex 3D scenes with a high density of objects, ensuring physical accuracy and realism, which helps improve robot performance in real-world scenarios.
Engineers at MIT and EPFL have developed a flapping-wing robot, or FAAV, that can swim underwater and then flap into the air, mimicking the movement of diving birds. The robot, weighing less than 300 grams, can transition between water and air using optimized wing size, flapping frequency, and tail angle. The study, published in Science, could enable new aerial-aquatic drones for oceanographic research and environmental monitoring.
NVIDIA and Hugging Face are releasing the NVIDIA Isaac GR00T 1.7 model and Isaac Teleop framework into LeRobot, an open-source robotics library, to provide developers with shared tools for training, evaluating, and deploying robot foundation models. NVIDIA Cosmos 3, a frontier world model for physical AI, is also set to be integrated soon.
LeRobot v0.6.0 introduces world model policies that learn to imagine the future, new VLAs, and a reward models API. It also includes six new simulation benchmarks, improved dataset handling with depth support and language annotations, and deployment tools like lerobot-rollout.
Apptronik announced the opening of its expanded Robot Park in Austin, Texas, and the launch of Apollo 2, a modular humanoid robot platform. The facility and robot are part of a global network, enabling real-world data collection and training for embodied AI models. Apollo 2 supports both bipedal and wheeled configurations, with data used to refine Gemini Robotics models for commercial deployment.
Firefly Aerospace's Blue Ghost Mission 2 will deploy NVIDIA Jetson for on-orbit AI processing, enabling real-time lunar data analysis and reducing latency. This marks the first use of Jetson in lunar orbit, supporting scientific research and future lunar exploration.
1X has launched the 1X World Model Lab, a new research organization dedicated to accelerating the development of fully autonomous humanoids through large-scale embodied world model pretraining. The lab is led by Sam Sinha, a former research scientist at Luma AI, and focuses on integrating diverse data sources to build robust and generalizable models. The initiative aims to close the full learning loop from data curation to real-world deployment.
Sanctuary AI has demonstrated zero-shot in-hand manipulation using their proprietary hydraulic hand, autonomously reorienting a lettered cube ten times without dropping it. This showcases successful simulation-to-reality transfer, overcoming challenges in simulating dexterous hands and highlighting the company's advanced robotic hardware and policy training capabilities.
Dexterity introduces Foresight, a world model that enables robots to reason about the physical world, predict outcomes, and act confidently in real environments. Trained on over 100 million autonomous actions in production, Foresight is designed for interpretability, safety, and performance, with capabilities like predictive branching and pragmatic decision-making. It addresses complex tasks like spatial packing and includes a game and API challenge for further exploration.
Physical AI refers to AI systems that enable robots to perform physical tasks in the real world. The article explains that this is extremely challenging due to the infinite variability of the physical environment, requiring solutions across perception, planning, motion control, and safety. Dexterity's compositional approach, using specialized AI agents and an orchestrator, enables production-scale Physical AI with over 100 million autonomous actions executed in real enterprise operations.
Dexterity introduces 'Transactable World Models' as a foundational component for Physical AI, emphasizing interpretability, physics integration, and real-time consistency. The approach treats world models as operators rather than data stores, enabling multi-agent coordination and robust manipulation in complex environments. This contrasts with traditional world models that focus on data compression and replay, lacking causal reasoning and physical grounding.
Dexterity, a leader in physical AI and robotics, has partnered with ASRock Rack to develop custom AI supercomputing systems for its Mech industrial superhumanoid robot. These edge AI servers provide real-time processing for warehouse automation tasks, enhancing autonomy and performance without cloud dependency.
This hands-on tutorial walks through the process of collecting data, training policies, and deploying autonomous medical robotics workflows on real hardware using NVIDIA Isaac for Healthcare. It introduces the SO-ARM starter workflow, which enables developers to build and validate surgical assistant robots from simulation to deployment.
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).