LeRobot/Cosmos3-Nano

Cosmos3-Nano base for LeRobot

This repository contains the LeRobot policy conversion of the NVIDIA Cosmos3-Nano base checkpoint. It is a starting point for robot policy post-training. The conversion adds no task-specific fine-tuning.

The policy jointly predicts robot action chunks and future video latents from language and current camera observations, with optional current-state conditioning. It contains the model components needed by the LeRobot policy interface.

The stored scalar count is not a count of trainable or active parameters. The package includes a Wan2.2 VAE, tokenizer files, and serialized LeRobot processors.

Loading and post-training

Use a LeRobot version containing the Cosmos3 policy types. Conversion and CPU loading checks used the exact commit above.

The saved observation and action features are empty. Frame rate and camera size are unresolved. A target dataset must provide these values before constructing the policy. Read the configuration with:

from lerobot.configs.policies import PreTrainedConfig

config = PreTrainedConfig.from_pretrained("lerobot/Cosmos3-Nano")

The training entry points select different configurations:

• --policy.path=lerobot/Cosmos3-Nano preserves the saved DROID post-training recipe, including domain 8, 32-step chunks, state conditioning, identity normalization, UniPC with 4 inference steps, and guidance 3. State and action must already use the same layout and width.
• --policy.type=cosmos3_nano --policy.pretrained_path=lerobot/Cosmos3-Nano uses these weights with the size class's default LIBERO recipe, including domain 5, 16-step chunks, no state conditioning, action quantile normalization, UniPC with 30 steps, and guidance 1. The dataset must match that recipe's camera and action layout.

load_action_modules defaults to true, preserving the released action modules. For native-style post-training, --policy.load_action_modules=false reinitializes the action input projection, action output projection, and action modality embedding. The setting returns to true after loading so subsequent reloads preserve trained action modules.

The saved dtype=float32 supports FP32 master weights during training. Most stored weights retain upstream BF16 precision. The four time-embedder tensors are FP32. For inference set --policy.dtype=bfloat16. For evaluation also set --tf32=false.

Scope and evaluation

The LeRobot implementation supports joint video-and-action wam mode, a single observation step, and the UniPC sampler. Actual action width comes from the target dataset. This package does not expose the upstream reasoning, audio, or standalone video-generation APIs.

CPU checks verified conversion and strict loading with resolved dataset features. No LIBERO closed-loop success rate is reported for this base conversion. Results for separately fine-tuned LIBERO checkpoints or NVIDIA Policy-DROID models do not measure this checkpoint.

License and provenance

The NVIDIA model materials use OpenMDW 1.1. Retain the license and applicable copyright and origin notices when redistributing them. NOTICE records the original model and conversion attribution. Separately installed runtime software retains its own licenses.

Consult the original NVIDIA model card and its safety, bias, privacy, and explainability notes for upstream limitations.