Rho: A Foundation for Efficiently Adaptable VLA Models

Overview

Adaptation is the bottleneck. Rho is built for it.

Vision-language-action models (VLAs) promise general-purpose robot manipulation, but their zero-shot success on unseen tasks and in new environments is still low. In practice, adaptation — turning a pretrained model into one that works on a particular robot for a particular task — is the crucial step, and it is data-hungry.

The reason is that pretraining exposes a model to a broad but shallow mix of robots, tasks, and environments. It never masters the sensing, kinematics, action space, and data conventions of any one platform. So when a pretrained model is finetuned on a new task, the demonstrations have to teach it the robot and the task at the same time. Rho's premise is that a model should master the robot before task adaptation begins. Its training recipe separates adaptation to an embodiment from adaptation to a task, and absorbs the first, expensive part into the released checkpoints.

01

One foundation, three robot-ready variants

A single pretrained model, Rho-base, is midtrained into Rho-YAM-Box, Rho-UR-AI-Trainer, and Rho-FR3-Duo — reusable starting points for many downstream tasks on each robot. All four are open weights.

02

Midtraining halves the task data you need

In controlled experiments, a midtrained variant reaches a given success rate with half as much finetuning data as Rho-base. After finetuning on the same demonstrations, the variants match or outperform π0.5, GR00T N1.7, and MolmoAct2 on three physical robots.

03

Keeps learning after deployment

A lightweight latent policy inside every Rho model learns from corrective feedback while the rest stays frozen. Online adaptation with FlowDAgger on just 15 episode corrections lifted test-tube assembly on FR3 Duo from 30% to 70% success on the hardest task configurations.

Training recipe

Regularized progressive adaptation, stage by stage

Rho task policies are produced by a recipe consisting of four offline stages and an optional fifth that runs after deployment. Each stage adds a capability — physical understanding, continuous control, one robot's conventions, one task, then corrections from use — while taking care not to erase what the previous stages built: vision-language batches stay in the mix through pretraining and midtraining, and online adaptation touches only a small internal module. That is the "regularized" in regularized progressive adaptation.

01Ground

Phi-Phy

Physically grounded VLM

A 4.68B-parameter Phi-family vision-language model gets extra robotics-relevant grounding, pointing, spatial-reasoning, and multi-image VQA training, so it understands physical scenes before it is asked to control anything.

Data

≈36M physical-grounding records in a 100M-record SFT mix

Mixture

General VLM data + physical grounding

Output

Phi-Phy backbone (no actions yet)

02Pretrain

Rho-base

Multi-embodiment VLA

A 542M-parameter flow-matching action expert is attached and trained with the backbone on Rho-Tomyum: 3,200+ hours from more than a dozen robots, mostly dual-arm, plus VQA data that keeps the backbone grounded.

Data

3,200+ robot hours · 48 B200 GPUs

Mixture

90% robot data · 10% vision-language

Output

Rho-base: the shared foundation

03Midtrain

Rho-⟨robot⟩

Embodiment-specific VLA

Rho-base is trained for two epochs on ~300 hours of one robot's data across dozens of task families. The goal is to learn the robot — cameras, kinematics, workspace, control conventions — not to master any one task.

Data

≈280–300 hours of one robot · 19–65 task families

Mixture

90% target robot · 10% vision-language

Output

Rho-YAM-Box · Rho-UR-AI-Trainer · Rho-FR3-Duo

04Finetune

Task policy

Offline task adaptation

A midtrained variant is finetuned on demonstrations of the target task: same flow-matching objective, all parameters trainable, robot data only. Because the robot is already familiar, a few hundred demonstrations go a long way.

Data

≈100–300 demonstrations per task on physical robots

Mixture

100% target-task demonstrations

Output

Deployable task policy

05Correct

Online adaptation

Learning from corrections

After deployment, a small latent policy inside the model learns from human corrective takeovers while the backbone and action expert stay frozen. It changes only where action generation starts, so the offline-learned behavior is preserved.

Data

15 corrected episodes per task on FR3 Duo

Mixture

Corrective takeovers only; rest of the model frozen

Output

The stage-04 task policy, improved in place

Stage 02 · What Rho-base is pretrained on

The Rho-Tomyum mixture. Shares are of training updates: 90% robot and UMI-style data, 10% vision-language data. All robot data is bimanual except the small OXE "Amuse Bouche" subset. Every source is standardized to chunk-relative end-effector deltas with 6D rotations (10 dimensions per arm) over about one second of motion, and demonstration segments labeled as errors are removed.

Stage 03 · What each variant is midtrained on

Rho-YAM-Box65 task families · 2,617 instructions

Lab and in-the-wild YAM Box data, sampled evenly at the episode level.

Rho-UR-AI-Trainer31 task families · 40,220 instructions

Physical demonstrations plus Isaac Sim rollouts on the same robot and cameras.

Rho-FR3-Duo19 task families · 33,883 instructions

Cleaning, folding, sorting, pouring, packing, cable insertion.

Each mixture averages no more than ~15 hours per task family — not enough to master a high-precision task, but enough to cover a robot's motions, workspace, and camera views. Instruction counts include task and subtask descriptions; the vision-language share is the same RoboPoint and RefSpatial data as in pretraining, 5% each.

Architecture

A compact Phi-family VLM coupled to a flow-matching action expert

At each control step Rho sees one or more camera views, a language instruction, and the robot's proprioceptive state, and outputs a chunk of future actions. Images and text are processed by Phi-Phy; robot state is projected straight into the action expert. The expert learns a velocity field that carries Gaussian noise to an action chunk and integrates it in 10 Euler steps at inference. Every expert block cross-attends to the same intermediate Phi-Phy representation (decoder block 14), which is computed once per query and reused across all flow steps.

The interface is deliberately generic: continuous state and action vectors of up to 32 dimensions, masked when unused, so the same model supports joint-space or end-effector control, delta or absolute targets, and one or two arms. The chunk length and control rate are properties of a training recipe, not of the architecture.

Phi-Phy backbone

Action expert

Phi-Phy starts from Microsoft's Phi family, chosen for capability per parameter. Its physical grounding adds public robotics-relevant sets (MGrounding, RefSpatial, XVR, RoboPoint), multi-image grounding, pointing, OCR, and counting data, and synthetic difference-spotting and odd-one-out questions to the general vision-language mix.

Embodiment midtraining

Midtraining halves the task data a robot needs

Two controlled experiments isolate what the midtraining stage buys. The first varies the amount of task data and holds compute fixed; the second holds the task data fixed and varies the amount of midtraining.

• Rho-base (pretrained only)
• Rho-RoboTwin (midtrained)

Success on the five held-out RoboTwin tasks (50 episodes per task) after 40k finetuning steps. Bars are means over seeds and whiskers show ± one standard deviation; the midtraining advantage is largest with little data and shrinks as the finetuning set grows.

60.0%Pretrained onlyRho-base finetuned directly

76.7%1 midtraining epochSame task data and budget

86.7%2 midtraining epochsThe released Rho-FR3-Duo

Bimanual plug insertion on FR3 Duo: success over 30 trials, with identical task data and finetuning budget in every condition.

Experiments on physical robots

Finetuned Rho variants vs. π0.5, GR00T N1.7, and MolmoAct2

The central experiments of the report finetune each midtrained Rho variant and the strongest open-weights VLAs on the same task demonstrations, then evaluate them side by side on the physical robot. Baselines start from their own pretrained checkpoints; MolmoAct2 is included on YAM Box because its training mix contains 720 hours from a similar dual-arm YAM robot. Rollouts follow a paired A/B protocol with matched initial configurations, 30 per task (60 on YAM Box), and no human intervention during a scored trial.

Every video in this section plays in real time. None is sped up.

I²RT

YAM Box

2 × 6-DoF YAM arms · parallel-jaw grippers · 3 RealSense cameras.

Universal Robots

UR AI Trainer

2 × UR5e arms · Robotiq 2F-85 grippers · 3 Orbbec cameras.

Franka

FR3 Duo

2 × 7-DoF FR3 arms · Robotiq grippers · ZED + 2 RealSense cameras.

On every platform Rho commands bimanual Cartesian end-effector targets (position, 6D orientation, gripper) from three RGB views and the two arm poses; the robot's controller turns them into joint commands.

YAM Box · high-data regime

BusyBox: strong performance with plentiful data

Rho-YAM-Box, π0.5, GR00T N1.7, and MolmoAct2 were each finetuned on about 2,000 demonstrations spanning the six task categories of the BusyBox benchmark — pushing illuminated buttons, flipping switches, moving sliders, turning a knob, pulling wires, and rotating or sliding the whole box — and evaluated over 60 rollouts with varied instructions and box poses. Rho-YAM-Box reaches 90% overall success, tied with π0.5 and ahead of GR00T N1.7 (53%) and MolmoAct2 (43%).

This result validates an important point: when finetuning data is copious, Rho's architecture and training recipe enable adaptation that is at least as robust as that of the strongest open-weights VLAs. π0.5 and MolmoAct2 are already familiar with YAM-Box-like robots from training on large amounts of Aloha-style and dual-YAM demonstrations; even so, finetuned Rho-YAM-Box is on par with π0.5 and ahead of MolmoAct2. Put differently, nothing in Rho's recipe prevents its finetuned policies from reaching high success rates when the dataset affords it. The finetuning dataset is released as microsoft/BusyBox.

Download rho-yam-box ↗

• Rho-YAM-Box
• π0.5
• GR00T N1.7
• MolmoAct2

Success rate on the six BusyBox task categories and overall; 10 rollouts per category per model.

UR AI Trainer · low-data regime

Multi-stage tasks from ~150 demonstrations

Two long-horizon tasks, one policy per task: toolbox packing (~150 demonstrations; lift a tray, fit it into a toolbox without spilling, close the lid, engage the latch) and small-electronics cleanup (~160 demonstrations; pick two gripper-tip-sized components and place each into an empty tray compartment). Rho-UR-AI-Trainer reaches 53.4% success and 70.0% mean task progress overall, roughly double π0.5 (30.0% success) and well above GR00T N1.7 (15.0%).

Neither baseline has seen much dual-arm UR data, so their finetuning must learn the robot and the task together — the situation midtraining is designed to avoid. On electronics cleanup, π0.5 nearly matches Rho's task progress: it tends to fail at the final placement, while Rho's rarer failures happen at the pick.

Download rho-ur-ai-trainer ↗

• Rho-UR-AI-Trainer
• π0.5
• GR00T N1.7

Success rate (left) and mean task progress (right) over 30 rollouts per task. Task progress credits each completed stage of a task.

FR3 Duo · precision and coordination

Five contact-rich task variants, 80% success

Three finetuning datasets yield five evaluation variants: plug insertion (~150 demonstrations), tumbler stacking and unstacking (~100), and test-tube racking and unracking (~300). Rho-FR3-Duo achieves 80.0% success and 88.7% task progress overall, against 50.0% / 71.3% for GR00T N1.7 and 46.7% / 64.5% for π0.5, and leads on every variant. A likely explanation: a task's finetuning demonstration set covers only a narrow slice of the robot's workspace, so a model finetuned from a general checkpoint must infer the robot's reach and motion geometry from that slice while learning the task. Midtraining has already shown Rho-FR3-Duo a far broader range of poses and motions, which gives it a stronger start when, at deployment time, objects sit near the edge of, or outside, the configurations in the finetuning data.

Stacking and racking are harder than their reverses for every model, because they require holding precise alignment under contact. Rho's largest margins are exactly there: +33.3 points on tumbler stacking and +30.0 on racking over the best baseline, plus +20.0 on plug insertion.

Download rho-fr3-duo ↗

• Rho-FR3-Duo
• π0.5
• GR00T N1.7

Success rate (top) and mean task progress (bottom) over 30 matched initial configurations per variant.

Online adaptation

After deployment, Rho keeps learning from corrections

A policy finetuned offline is only as good as the states in its demonstrations; once deployed, it will meet configurations where it is unreliable. Rho addresses this with an optional internal latent policy. Normally the action generator G starts its flow from Gaussian noise z; with the latent policy enabled, it starts from a learned, observation-conditioned z = π(o) instead, so a = G(o, π(o)). The backbone and action expert stay frozen; only this small module is trained, and it is stored and served inside the same checkpoint.

Training data comes from corrective takeovers (the FlowDAgger recipe): when a human — or, in simulation, a scripted expert — takes over, the actions actually executed are assembled into a chunk, inverted through the frozen flow to recover the latent that would have produced them, and used as a regression target for π. Online learning therefore changes where generation starts, while every action is still produced by the unchanged offline-trained model.

Meta-World (simulation)

A deliberately under-trained Rho policy on all 50 Meta-World tasks, adapted on six hard tasks with 50 scripted-expert episodes each. Success over 100 rollouts per task.

FR3 Duo (physical robot)

Rho-FR3-Duo policies finetuned offline on 150 demonstrations per task, then adapted from 15 human-supervised episodes — a tenth of the offline budget — and evaluated on 10 hard configurations at the edge of the workspace, three trials each.

Corrections on FR3 Duo are given with a SpaceMouse while the operator keeps emergency-stop authority; the learned policy can only change a bounded latent, and every generated action remains subject to the controller's joint, workspace, and action limits.

Simulation benchmarks

Rho-base on LIBERO and RoboEval

The simulation comparisons start from Rho-base rather than a midtrained variant and follow each benchmark's standard protocol, finetuning on the benchmark's own data. Rho's policies are adapted for 40k steps at batch size 128; success is averaged over three evaluation seeds or passes.

Success rates in percent. Baseline values are taken from each model's report or official release, so adaptation and evaluation protocols may differ; Rho is the mean over three seeds.

• Rho-base
• π0.5
• GR00T N1.7
• MolmoAct2

RoboEval task-level and overall success for Rho-base, π0.5, GR00T N1.7, and MolmoAct2. Whiskers show ± one standard deviation over three evaluation passes; hover a bar for its value.

RoboEval success rates as a table

0.85Task progressionHighest of the compared models

1.41 mCartesian path lengthShortest of the compared models

7.70 / 4.69 radJoint / orientation path lengthShortest of the compared models

Beyond success, RoboEval reports behavioral metrics. Rho leads on five of nine; π0.5 has the fewest environment collisions, and MolmoAct2 the lowest gripper-height difference, self-collision count, and slip count.

Ablations

Why Rho is built the way it is

Architecture design choices were selected on RoboEval, where each candidate was pretrained on a 15% subset of Rho-Tomyum and then adapted for 40k steps; success is the mean over eight tasks and three evaluation passes. Two dimensions mattered: 128-dimensional attention heads beat both smaller and larger heads at either width, and width 2,048 beat 1,024 even when the narrower expert was made twice as deep. The gains concentrate on contact and precision tasks — lifting a flat book, stacking blocks, shelving a book — where successful action chunks occupy a narrow region of action space and the velocity field must be resolved finely around it.

Shape sweep (RoboEval success after 40k adaptation steps)

Compression (width 2,048 and head dim. 128 kept fixed)

Sharing the adaLN modulation maps, grouped-query attention, and a 12-block depth cut the expert from 1.65B to 542M parameters — about two thirds — at the same success rate. Routing successive blocks to different backbone layers was also tried and was worse (60.5% vs 67.3%), so every block reads decoder block 14.

Pretraining recipe

Keeping vision-language batches in the pretraining mix helps most on tasks with semantic and temporal structure: on LIBERO at 20k adaptation steps it lifts the Goal suite from 94.7% to 97.7% and the Long suite from 92.7% to 96.0% relative to robot-only pretraining. A learning-rate sweep for the action expert selected 10⁻⁴; the choice affects the quality of the transferred initialization, not just early adaptation speed.

67.3%Robot + vision-language pretrainingRho's recipe

65.2%Robot-only pretrainingSame robot data, no VQA batches

61.0%No pretrainingGrounded backbone, fresh action expert

RoboEval success rate after 40k adaptation steps; the pretrained variants use the same 15% subset as the ablations above. Pretraining on robot data lifts success by about four points over a fresh action expert, and keeping vision-language batches in the mix adds two more.

Code and paper

Citation

If you use Rho in your research, please cite the technical report:

@article{rhoteam2026rhofoundationefficientlyadaptable,
      title={Rho: A Foundation for Efficiently Adaptable VLA Models},
      author={Rho Team and Simran Bagaria and Daphne Chen and Dean Fortier and Jianlong Fu and Michael Harrison and Tess Hellebrekers and Neel Joshi and Andrey Kolobov and Dalton Moore and Galen Mullins and Michael Murray and Eduardo Salinas and Reuben Tan},
      year={2026},
      journal={arXiv preprint arXiv:2609.38164},
      url={https://arxiv.org/abs/2609.38164}
}

Models

Start from a midtrained checkpoint if you have one of the three robots; start from Rho-base to midtrain a new platform or to adapt to a different one directly.

Scope and responsible use. The strongest-model claims hold for the baselines, budgets, and protocols in the report; results on three platforms and a handful of benchmarks do not establish universal generalization. Offline adaptation still depends on representative demonstrations, and online adaptation should run under human supervision with platform-specific safeguards, workspace constraints, and evaluation gates before deployment.