LeRobot/pi05_base: Vision-Language-Action Model for Open-World Generalization

No manual editorial approval recorded.

Original source quotation: “π₀.₅ (Pi05) (LeRobot) π₀.₅ is a Vision-Language-Action (VLA) model with open-world generalization from Physical Intelligence, co-trained on robot demonstrations and large-scale multimodal data to execute long-horizon tasks in unseen real-world environments.”

Source E1

Original source quotation: “Note: This model currently supports only the flow-matching action head for π₀.₅ training and inference. Other components from the original work (e.g., subtask prediction, action tokenization, or RL) were not released upstream and are not included here, though the LeRobot team is actively working to support them.”

Source E2

Original source quotation: “Original paper: π0.5: A Vision-Language-Action Model with Open-World Generalization Reference implementation: https://github.com/Physical-Intelligence/openpi LeRobot implementation: Follows the original reference code for compatibility.”

Source E3

Original source quotation: “Model description Inputs: images (multi-view), proprio/state, optional language instruction Outputs: continuous actions Training objective: flow matching Action representation: continuous Intended use: Base model to fine tune on your specific use case”

Source E4

Original source quotation: “Quick start (inference on a real batch) Installation pip install "lerobot[pi]@git+https://github.com/huggingface/lerobot.git" For full installation details (including optional video dependencies such as ffmpeg for torchcodec), see the official documentation: https://huggingface.co/docs/lerobot/installation”

Source E5

Original source quotation: “Load model + dataset, run select_action import torch from lerobot.datasets.lerobot_dataset import LeRobotDataset from lerobot.policies.factory import make_pre_post_processors # Swap this import per-policy from lerobot.policies.pi05 import PI05Policy # load a policy model_id = "lerobot/pi05_base" # <- swap checkpoint device = torch.device( "cuda" if torch.cuda.is_available() else "cpu" ) policy = PI05Policy.from_pretrained(model_id).to(device). eval () preprocess, postprocess = make_pre_post_processors( policy.config, model_id, preprocessor_overrides={ "device_processor" : { "device" : str (device)}}, ) # load a lerobotdataset (we will replace with a simpler dataset) dataset = LeRobotDataset( "lerobot/libero" ) # pick an episode episode_index = 0 # each episode corresponds to a contiguous range of frame indices from_idx = dataset.meta.episodes[ "dataset_from_index" ][episode_index] to_idx = dataset.meta.episodes[ "dataset_to_index" ][episode_index] # get a single frame from that episode (e.g. the first frame) frame_index = from_idx frame = dict (dataset[frame_index]) batch = preprocess(frame) with torch.inference_mode(): pred_action = policy.select_action(batch) # use your policy postprocess, this post process the action # for instance unnormalize the actions, detokenize it etc.. pred_action = postprocess(pred_action)”

Source E6

Original source quotation: “Training step (loss + backward) If you’re training / fine-tuning, you typically call forward(...) to get a loss and then: policy.train() batch = dict (dataset[ 0 ]) batch = preprocess(batch) loss, outputs = policy.forward(batch) loss.backward()”

Source E7

Original source quotation: “Notes: Some policies expose policy(**batch) or return a dict; keep this snippet aligned with the policy API. Use your trainer script ( lerobot-train ) for full training loops.”

Source E8

Original source quotation: “How to train / fine-tune lerobot-train \ --dataset.repo_id= ${HF_USER} /<dataset> \ --output_dir=./outputs/[RUN_NAME] \ --job_name=[RUN_NAME] \ --policy.repo_id= ${HF_USER} /<desired_policy_repo_id> \ --policy.path=lerobot/[BASE_CHECKPOINT] \ --policy.dtype=bfloat16 \ --policy.device=cuda \ --steps=100000 \ --batch_size=4 Add policy-specific flags below: -policy.chunk_size=... -policy.n_action_steps=... -policy.max_action_tokens=... -policy.gradient_checkpointing=true”

Source E9

Original source quotation: “Real-World Inference & Evaluation You can use the record script from lerobot-record with a policy checkpoint as input, to run inference and evaluate your policy. For instance, run this command or API example to run inference and record 10 evaluation episodes: lerobot-record \ --robot.type=so100_follower \ --robot.port=/dev/ttyACM1 \ --robot.cameras="{ up: {type: opencv, index_or_path: /dev/video10, width: 640, height: 480, fps: 30}, side: {type: intelrealsense, serial_number_or_name: 233522074606, width: 640, height: 480, fps: 30}}" \ --robot.id=my_awesome_follower_arm \ --display_data=false \ --dataset.repo_id=${HF_USER}/eval_so100 \ --dataset.single_task="Put lego brick into the transparent box" \ # <- Teleop optional if you want to teleoperate in between episodes \ # --teleop.type=so100_leader \ # --teleop.port=/dev/ttyACM0 \ # --teleop.id=my_awesome_leader_arm \ --policy.path=${HF_USER}/my_policy”

Source E10