Technical Directions
Robotics coverage relating to sim-to-real.
152 selected reportsLast 30 days 121 recordsAll coverage 565 records
1X has launched the 1X World Model Lab, a new research organization dedicated to accelerating the development of fully autonomous humanoids through large-scale embodied world model pretraining. The lab is led by Sam Sinha, a former research scientist at Luma AI, and focuses on integrating diverse data sources to build robust and generalizable models. The initiative aims to close the full learning loop from data curation to real-world deployment.
Sanctuary AI has demonstrated zero-shot in-hand manipulation using their proprietary hydraulic hand, autonomously reorienting a lettered cube ten times without dropping it. This showcases successful simulation-to-reality transfer, overcoming challenges in simulating dexterous hands and highlighting the company's advanced robotic hardware and policy training capabilities.
Dexterity introduces Foresight, a world model that enables robots to reason about the physical world, predict outcomes, and act confidently in real environments. Trained on over 100 million autonomous actions in production, Foresight is designed for interpretability, safety, and performance, with capabilities like predictive branching and pragmatic decision-making. It addresses complex tasks like spatial packing and includes a game and API challenge for further exploration.
Physical AI refers to AI systems that enable robots to perform physical tasks in the real world. The article explains that this is extremely challenging due to the infinite variability of the physical environment, requiring solutions across perception, planning, motion control, and safety. Dexterity's compositional approach, using specialized AI agents and an orchestrator, enables production-scale Physical AI with over 100 million autonomous actions executed in real enterprise operations.
Dexterity introduces 'Transactable World Models' as a foundational component for Physical AI, emphasizing interpretability, physics integration, and real-time consistency. The approach treats world models as operators rather than data stores, enabling multi-agent coordination and robust manipulation in complex environments. This contrasts with traditional world models that focus on data compression and replay, lacking causal reasoning and physical grounding.
Dexterity, a leader in physical AI and robotics, has partnered with ASRock Rack to develop custom AI supercomputing systems for its Mech industrial superhumanoid robot. These edge AI servers provide real-time processing for warehouse automation tasks, enhancing autonomy and performance without cloud dependency.
This hands-on tutorial walks through the process of collecting data, training policies, and deploying autonomous medical robotics workflows on real hardware using NVIDIA Isaac for Healthcare. It introduces the SO-ARM starter workflow, which enables developers to build and validate surgical assistant robots from simulation to deployment.
Hugging Face announces LeRobot v0.4.0, a major upgrade for open-source robotics with Dataset v3.0, new VLA models like PI0.5 and GR00T N1.5, and a plugin system for hardware integration. The release also adds support for LIBERO and Meta-World simulations, multi-GPU training, and a new Hugging Face Robot Learning Course.
Hugging Face introduces SmolVLA, a 450M parameter open-source Vision-Language-Action model for robotics, trained on community-shared datasets. It outperforms larger models in simulation and real-world tasks, supports asynchronous inference for faster response, and is designed for deployment on consumer hardware.
SmolVLA is a compact, efficient Vision-Language-Action (VLA) model designed for affordable robotics, trainable on a single GPU and deployable on consumer hardware. It matches the performance of larger VLAs through community-driven data, with a reference implementation and training instructions provided.
Sanctuary AI has demonstrated industry-leading sim-to-real transfer of dexterous manipulation policies using their high-degree-of-freedom hydraulic hands. The company leverages NVIDIA Isaac Lab, an open-source framework for robot learning, to simulate dexterity-focused training environments. This approach enables efficient training of autonomous robots, with the ability to train thousands of hands simultaneously and accelerate the learning process through online reinforcement learning.
NIST is launching the Agile Robotics for Industrial Automation Competition (ARIA) to improve robot agility in manufacturing. The competition, a joint effort with the IEEE Conference on Automation Science and Engineering, will focus on tasks such as failure recovery, automated planning, and plug-and-play robot integration. The goal is to reduce programming costs and increase adaptability for small and medium manufacturers.
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).