Fine-Grained Annotated Robotic Trajectory Dataset and Vision-Language-Action Policy

\papertype

Scale AI Research \secondlogopathassets/hf-logo-with-title.png \secondlogoheight0.32in \headerlogoheight0.22in \titlefontsize1923

^††footnotetext: *Equal contribution.  † Equal senior contribution.  ‡ Work done while at Scale AI.

   Pepijn Kooijmans Affiliation: Hugging Face   
Mansi Agarwal Affiliation: Scale AI    Yusuf Umut Ciftci Affiliation: Scale AI Affiliation: University of Southern California Affiliation: Stanford University    Aseem Doriwala Affiliation: Scale AI   
Catherine Weaver Affiliation: Scale AI    Mouli Sivapurapu    Kai Yang Affiliation: Scale AI   
Thomas Wolf Affiliation: Hugging Face    Jackson Lee Affiliation: Scale AI    Pragna Mannam Affiliation: Scale AI

Abstract

Robots operating in real-world environments must often execute complex, multi-step bimanual tasks over long horizons rather than single, isolated actions. Current manipulation datasets struggle to support this capability: although single-arm datasets reach hundreds of thousands of trajectories, they typically provide only one high-level instruction per episode, while existing bimanual datasets with subtask labels annotate only part of their recorded hours. We present FineART, a densely annotated bimanual manipulation dataset comprising 40,543 episodes (1,718 hours) and 533,913 subtasks across 151 tasks. We also introduce FineART-VLA, a vision-language-action policy that predicts its own next subtask to guide its actions. Mid-training on FineART’s subtask annotations raises FineART-VLA’s success at following spatial instructions from 32.0% to 100.0%. With step-by-step human subtask guidance, it also raises success on unseen long-horizon tasks from 16.0% to 76.0%. Furthermore, after minimal fine-tuning on a new robot, the policy requires only one-tenth of the data needed by baselines without this mid-training and generalizes zero-shot to tasks unseen on the new hardware. We open-source the full dataset, model weights, and training code.

1 Introduction

“Divide each difficulty into as many parts as is feasible and necessary to resolve it.”

— René Descartes, Discourse on Method (1637), Part II

Robotic assistants that work in real households and workplaces must handle long-horizon tasks – such as preparing a meal, sorting tools, or packing a bag – that demand dozens of coordinated two-handed actions over several minutes. Humans manage this complexity by continuously breaking a task into a running sequence of subtasks (e.g., pick up plate, pick up toast, put toast on plate) instead of holding one instruction for the full duration. Giving a robot policy that same ability requires two foundations: (1) long-horizon bimanual manipulation data and (2) language supervision that marks subtask boundaries densely and persistently enough to train on directly.

While single-arm datasets [28, 20] scale to hundreds of thousands of trajectories, they consist mainly of short pick-and-place episodes associated with single instructions. Recent bimanual datasets [36, 13, 12] address embodiment gaps and scale, but even the largest [2] only annotates subtask labels for a fraction of its hours. Furthermore, they report diversity in aggregate, obscuring whether they cover contact-rich, long-tail skills like insertion, deformable object manipulation, or tool use. Existing annotation pipelines [26, 1, 3] make this worse: they either assign broad, high-level labels to entire videos or try to generate subtasks on the fly during execution, rather than giving the model step-by-step guidance directly throughout training.

Without step-by-step labels during training, current models struggle to handle long tasks. Flat vision-language-action models [6, 22] map an instruction and image directly to actions, with no way to track progress through a multi-minute task, while hierarchical alternatives [1, 31] split planning and execution across two separate models. No existing dataset-and-model combination offers persistent, dense subtask training for long-horizon bimanual manipulation inside a single end-to-end policy. FineART closes this gap: a bimanual manipulation dataset where every one of its 1,718 hours across 40,543 episodes (averaging 152.5 seconds each) carries dense subtask annotations – 533,913 subtask labels in total, roughly 13 per episode – paired with a training recipe that treats these subtasks as persistent supervision inside a single end-to-end policy.

Our contributions are:

• •

  FineART Dataset. An open-source bimanual manipulation dataset with dense subtask annotations, containing 40,543 labeled episodes and 533,913 subtask labels.

• •

  FineART-VLA. A single end-to-end policy, extended from a pretrained vision-language-action backbone, that pairs System-2 subtask prediction in language with System-1 continuous-action execution using hierarchical inference and knowledge insulation instead of splitting planning and execution across two separate models.

• •

  Dense Subtask Training. Dense subtask training improves spatial grounding and long-horizon instruction following, raising success rate from 32.0% to 100.0% on a spatial disambiguation task and from 16.0% to 76.0% on a long-horizon task requiring human corrections, compared to a matched policy trained without subtask training.

• •

  Cross-Embodiment Mid-Training. Mid-training on FineART enables zero-shot transfer to unseen tasks – after fine-tuning on only five atomic tasks, the policy generalizes to unseen, multi-stage tasks on a distinct robot platform (YAM), reaching 28.0% success on prepare breakfast, 12.0% on sort tools, and 60.0% on put cable into bin under distractors.

We open-source the model and training code alongside the dataset.

2 Related Work

2.1 Large-Scale Robot Learning Datasets

Robot learning has scaled in waves. An early single-arm dataset [19] established teleoperated data collection at growing scale, but it stayed within a handful of tasks and seconds-long episodes. Later efforts [28, 20] pushed scale further still, aggregating over a million and 76k trajectories respectively across dozens of platforms and hundreds of in-the-wild scenes, yet both remain dominated by single-arm, short-horizon pick-and-place, with one instruction typically covering an entire episode. Bimanual datasets close part of that gap [36, 13, 12, 27]: two introduce low-cost teleoperation for contact-rich two-arm manipulation, one adds force and audio modalities across 110k sequences, and the last extends coverage to over 100 more tasks. Two very recent efforts push bimanual scale further still: MolmoAct2’s Bimanual YAM release [11] and the ABC-130K dataset [2], the latter also annotating a 1,552-hour subset with sub-episode task labels. Table 1 compares all of these datasets across trajectory count, task count, and annotated hours: FineART is the only dataset combining bimanual manipulation, multi-minute episodes, and dense sub-episode labels across its entire duration, rather than a curated subset of it.

2.2 Data Curation and Annotation Pipelines

Dense temporal annotation of long videos originated in captioning work [23], which jointly localized and described events in multi-minute footage. Robotics applied coarser versions of the same idea [26, 1], pairing subtasks with play data or decomposing an instruction into skills online rather than annotating the training data itself. A closer prior effort [3] inserted an intermediate language-motion layer between task and action, but its hierarchy was produced online rather than as a persistent, dataset-wide label. FineART instead annotates every episode directly, producing 533,913 subtask labels (an average of 13 per episode) that are available as fixed training targets rather than being generated at inference time.

2.3 Language-Conditioned Policies

Augmenting end-to-end policies with an explicit high-level reasoning step improved performance on long-horizon tasks [1, 31, 24, 14], particularly when the high-level step could draw on a pretrained LLM or VLM. Flat vision-language-action models [6, 22] skip this step, mapping a single instruction and image directly to actions; one extension [9] carries the same recipe to heterogeneous embodiments without unifying the action space. These architectures degrade over long horizons because the model does not track task progress. Most hierarchical methods instead split the two roles across separate models, with a VLM predicting semantic subtasks and a distinct low-level policy executing them [1, 31, 24]; others [25] conditioned on short natural-language subtasks mined from unstructured play data rather than a planner’s output. FineART instead uses a single model for both stages, closer to chain-of-thought [34] than to a two-model pipeline, built on a π0.5 flow-matching backbone [29] and knowledge insulation [10] to prevent continuous-action gradients from destroying pretrained VLM representations. Two concurrent efforts pursued closely related ideas: dense per-clip language annotation for policy learning [21], and a diffusion policy conditioned on foundation-model-generated subtasks [17]; a third [18] paired a VLM subtask planner with a flow-matching controller in a similar spirit. None reported a dataset combining bimanual multi-minute episodes with persistent sub-episode labels at FineART’s scale.

3 FineART Dataset

FineART is a bimanual manipulation dataset consisting of 40,543 teleoperated episodes (1,718 hours, 185.5 M frames) spanning 151 tasks. It is the largest subtask-annotated robot dataset to the best of our knowledge, providing full temporal supervision across 100% of its hours and tasks.

3.1 Data Collection Protocol

All data are collected on the Trossen Stationary AI bimanual leader–follower system by human teleoperators. Each follower arm carries an ALOHA2 gripper rated for a 1 kg payload and executes a 50 Hz control loop, with joint telemetry logged at 300 Hz. Visual state is recorded at 30 fps (1280×720) across four Intel RealSense D405 cameras: a bird’s-eye view for global tabletop context, a worm’s-eye view for low-angle contact physics, and dual wrist-mounted cameras for egocentric local manipulation.

A second independent team of human annotators labeled trajectories with natural task descriptions, without VLM assistance. Finally, a third independent team of human auditors performed quality control on a statistically significant sample of the labeled trajectories. This process ensures that the dataset is free of labeling bias from the teleoperators, and that the annotations are accurate, complete, and consistent.

Labeling yields two granularities (Fig. 3). Demonstration Label: Whole-episode text summary paired with metadata describing execution quality and outcome success. Subtask Labels: Contiguous, non-overlapping temporal segments containing untemplated free-text task descriptions.

3.2 Dataset Composition: Long Multistage Tasks

While single-arm datasets like Bridge V2 [33] and DROID [20] provide large trajectory volumes, they pair multiminute trajectories with a single instruction. ABC-130K [2] represents the only prior single-embodiment dataset providing timestamped subtask text alongside raw trajectories. Therefore, we use ABC-130K as a reference while analyzing our dataset composition.

FineART features long-horizon tasks that require multiple manipulation subtasks to complete, averaging 13.17 subtasks per task. The median episode duration is approximately 2.0 min, with a long tail of episodes lasting more than 10 min (Fig. 2d).

3.2.1 Subtask Composition

FineART deliberately concentrates most of its subtasks on a small set of reusable manipulation primitives, then adds a long tail of physically distinct, contact-rich skills on top. In a mid-training dataset, a policy needs to see a primitive many times to learn it reliably, while the contact-rich tail tests whether that learned representation actually carries over to harder, fine-grained tasks.

To capture manipulation diversity, we define a primary vocabulary of 31 leading verbs categorized into seven fundamental manipulation primitives (Fig. 2c).

While the majority of the dataset (1,326.2 h; 77.2% of hours) concentrates on gross pick-and-place (572.7 h) and semantic organization (753.5 h) to build core spatial manipulation abilities, the remaining 391.8 h (22.8% of hours) forms a long tail of contact-rich operations. This long tail covers tight tolerances, near-unbounded configuration states, and complex contact dynamics across deformable textile manipulation (126.0 h), tool-mediated assembly (111.9 h), flexible linear object routing (59.4 h), compliant alignment (50.8 h), and high-precision mechanical insertion (43.7 h).

Though smaller in total volume, this tail provides the essential demonstration signal needed for policies to move beyond simple prehension toward fine-motor, contact-guided behavior. Section 4 tests that hypothesis directly by ablating the tail from the training mixture.

We observe that FineART subtasks are about 2.5 times as long as ABC-130K subtasks, with median duration of 8.6 s versus 3.5 s (Fig. 2e), even though the datasets have similar arm-speed distributions (Fig. 2f). This indicates that FineART subtasks capture more multi-stage manipulation (stack, remove, close), while ABC-130K subtasks often segment simpler, individual actions (pass, grab, flatten).

3.2.2 Object Diversity

Object diversity is introduced during collection through object rotation and spatial randomization. Every five episodes, a unique set of objects is assigned to each task-operator pair. Objects are repositioned and reoriented for each episode so that no arrangement is repeated. This protocol varies both the objects used to perform tasks and the spatial configurations in which they must be manipulated.

To compare object diversity, we apply a shared rule-based parser to the subtask annotations of FineART and ABC-130K. After spelling normalization and typo correction, we extract noun phrases and represent object classes by singularized head nouns, retaining noun qualifiers for generic heads (e.g., chess piece). Preceding modifiers are retained separately as descriptors and tagged using lexicons for color, material, size, shape, pattern, state, spatial position, and ordinal attributes. The parser filters non-object expressions such as regions, robot parts, pronouns, and measurements. These statistics measure objects named in the annotations: a class is a normalized object label, and a mention is an occurrence of that label.

Figure 2a shows broad coverage across everyday object categories, spanning 657 distinct object classes in total. FineART contains more distinct labels than ABC-130K in food, clothing, receptacles, tools, electronics, and toiletries. Coverage also extends to stationery, tableware, parts, toys, and materials, with comparable coverage of surfaces and fewer labels for décor. Thus, the repeated manipulation primitives described above are demonstrated across a range of object identities, materials, and geometries. Because this comparison is annotation-based, it reflects both collection diversity and how specifically annotators name objects.

The Other category contains extracted labels that could not be mapped to a semantic category from the noun alone. FineART entries include ambiguous nouns such as stick and mesh, and parser leftovers such as form from the phrase “to form a loop.” In ABC-130K, approximately 95% of these mentions use generic labels such as personal item, distractor, chemistry item, and product.

3.2.3 Descriptive Specificity

The annotations describe not only which object class is involved, but also attributes that distinguish an object within a scene. Figure 2b reports descriptor frequency per object mention. Color modifiers occur in 81.1% of FineART mentions, compared with less than 0.01% in ABC-130K; spatial modifiers occur in 14.8% versus 0.80%. Material, size, ordinal, and pattern descriptors provide additional distinctions. ABC-130K uses state and shape modifiers more frequently, largely because of its many deformable-object demonstrations featuring descriptors such as folded, flattened, and crumpled.

This descriptive detail is reflected in the diversity of the annotation text: FineART contains 110,299 distinct subtask strings and 23,589 distinct object-referring expressions, compared with 7,977 and 534, respectively, in ABC-130K. In cases involving two objects of the same class, FineART annotations disambiguate the objects 81.4% of the time, compared with 44.0% in ABC-130K. Such distinctions let a subtask specify which instance to manipulate, beyond naming the action and object class.

Together, object variation and instance-specific language provide supervision for selecting and manipulating objects throughout multi-stage tasks. Section 4 evaluates how mid-training with these subtask annotations affects spatial grounding, instruction following, and transfer to a different embodiment.

4 Experiments

We evaluate FineART as a mid-training dataset for a pre-trained generalist VLA [29], mid-training for a relatively short run (300k steps) and then asking three questions (Sections 4.2 and 4.3). Does dense subtask training, together with knowledge insulation (KI), improve out-of-distribution generalization and long-horizon instruction following? How much does long-tail training data contribute to overall performance? And does FineART mid-training reduce the data required to adapt to a new embodiment? Over the course of our experiments, we execute 3,400 rollouts.

4.1 Policy and Mid-Training Setup

We start from π0.5 publicly released via LeRobot [29, 7], whose implementation does not support subtask training, knowledge insulation, or hierarchical subtask-then-action inference. We extend the architecture to add all three and refer to the resulting model as FineART-VLA: a System-2 subtask predictor and a System-1 continuous-action controller in one policy. It combines a 2B-parameter Gemma backbone [15] with a SigLIP vision encoder [35] and a separate 300M-parameter Gemma-style action expert. Images and language form one bidirectional attention block; the state/action suffix attends to itself and to that full block but not vice versa, and the subtask span being generated is further restricted to its own earlier tokens, matching autoregressive generation at inference. At step t the policy observes ot=(It1:3,qt), three of the rig’s camera streams (bird’s-eye and both wrists, resized to 224×224; the worm’s-eye view is recorded but not consumed) and the 14-dim bimanual joint and gripper states. The policy is conditioned on a language context ℓ that is either the task string ℓtask or a subtask ℓsub.

Following [29], the policy emits three outputs per forward pass: the next subtask in language, a FAST-tokenized action sequence a~, and a continuous chunk At=[at,…,at+H−1]∈ℝH×14 with H=50. Writing zθ≡zθ​(ot,ℓsub) for the backbone representation,

The continuous action head predicts a velocity vθ from the noised chunk Atτ=(1−τ)​At+τ​ϵ, trained by flow matching to minimize the L2-norm between vθ and the target field ϵ−At, while the discrete and text heads use next-token cross-entropy. The objective is

with λflow=10, λfast=1 and λtext∈{0,1}. Each action chunk in the dataset is represented in both the continuous flow target and the discretized token sequence. Including explicit task-to-subtask predictions (λtext>0) has been shown to improve performance on long-horizon tasks, even compared to human oracle subtask planning [29]. Setting λtext=0 removes the subtask channel, conditioning the policy on ℓtask directly.

Knowledge insulation (KI) [10] has been shown to mitigate the degradation of pre-trained VLM knowledge that occurs when a randomly initialized flow-based action expert is attached to the backbone. Insulating the backbone from ℒflow restores language-following, accelerates convergence by roughly 7.5×, and improves generalization to unseen objects and environments. With KI enabled, the action expert attends to stop-gradient copies of the backbone’s keys and values so that ∇θVLMℒflow=0 and the backbone is shaped only by the token-level losses.

When λtext=1, 30% of each batch is a sample (ℓtask, ℓsub, ot) for ℒtext and 70% is a sample (At, a~, ot, ℓsub) for ℒflow and ℒfast. All runs train for 300k steps at a global batch of 64, about 10% of one epoch over the 185.5M frames, using AdamW at 2.5×10−5 decayed to 5×10−6 in bfloat16. Mid-training draws frames with a task-balanced sampler following [29]: a task contributing ni frames is sampled with probability proportional to niα, so α=0 weights every task equally regardless of size (task-based sampling) and α=1 weights every frame equally across the pooled dataset (raw frequency). We use α=0.5, a blend of the two.

4.2 In-Embodiment Results (ALOHA)

We train policy ablations via Section 4.1 on the full FineART dataset and evaluate the policies on the same embodiment used for collection (Stationary Trossen ALOHA). For all experiments, each checkpoint is evaluated with 25 rollouts per task and is initialized with a prescribed set of 25 object locations, meaning all checkpoints are evaluated under the same initial conditions. We report success rate (SR) and progress rate (PR, the average fraction of task stages completed). At 25 rollouts, per-task SR is quantized to 4 points with a standard error near 9, and per-checkpoint averages carry roughly 3 to 4, indicating large, consistent effects rather than single-task differences.

4.2.1 Does dense subtask training and knowledge insulation (KI) improve OOD generalization?

We isolate the effect of subtask training (i.e., λtext=1) and KI with a 2×2 ablation that varies each independently: Run 1.1 (subtask training + KI), Run 1.2 (no subtask training + KI), Run 1.3 (subtask training, no KI), and Run 1.4 (no subtask training, no KI), each evaluated on the same suite of six tasks. The tasks are grouped in Table 2 by their relation to the tasks present in the FineART training set. Shelf and towel are in-distribution, with literal subtask strings. Vase and pegboard are partial, in that the insertion dynamics appear but the prompts differ. Saucer and sort tools are out-of-distribution: the dataset contains no saucer and no type-routed sorting of tools.

Comparing performance across in-distribution, partially in-distribution, and out-of-distribution tasks (Fig. 4), subtask training and knowledge insulation each improve overall performance. Policies with both subtask training and KI show the highest OOD generalization capabilities, indicating that both subtask training and knowledge insulation are necessary to maintain general pre-trained task knowledge.

With in-distribution tasks, performance is dependent on the type of task (Table 2). Shelf placement is a saturated, single-step task, and no subtask training and KI achieves the highest performance (96.0% vs. 72.0%) indicating that subtask training may hinder task performance, while KI is important to maintain task understanding. Conversely, towel folding, which is a more ambiguous task, benefits from subtask training more than KI. KI and subtask training both improve performance compared to the baseline in Partial ID tasks, but benefit the most from KI alone; the comparatively worse performance with subtask training may indicate an over-sensitivity to language conditioning.

4.2.2 How much does long-tail data contribute to overall performance?

Run 2.1 (Table 2b) removes the long-tail task categories defined in Fig. 2c from mid-training, keeping the same number of training steps; the retained set is only pick-and-place and sorting, 106 of 151 task labels (71.0% of valid frames), compared against Run 1.3 (full data, otherwise identical). Overall success drops from 24.7% to 14.0%. The simple shelf task is the exception, holding its SR steady (64.0% to 68.0%), but every other evaluation task degrades substantially when the long-tail is excluded, even though those tasks are nominally covered by the retained categories – indicating that long-tail diversity, not merely category coverage, drives generalization.

4.2.3 Does subtask training provide spatial grounding and long-horizon instruction following?

In Fig. 5, we further evaluate Run 1.3 (subtask training, no KI) against Run 1.4 (no subtask training, no KI) – KI fixed off in both arms to isolate subtask training alone – on two tasks designed to test spatial grounding or long-horizon instruction following. The experimental procedure is the same: 25 rollouts per task with prescribed initial conditions.

Spatial grounding. On put donut into the {left, right, middle} bin, the target is distinguished only by a spatial word in the instruction. Subtask training saturates the task with 100% SR. In contrast, the no-subtask policy only achieves 32% success and all 17 failures were due to placing the object in the incorrect bin. The results indicate that the dense subtask annotations in the FineART dataset are critical for spatial grounding and instruction following.

Long-horizon instruction following. Prepare breakfast is a multi-minute task whose subtasks are present in the dataset, but not the full task sequence. Figure 5 contrasts two inference conditions: autonomous execution, where the policy proposes its own subtasks (Run 1.3, hierarchical subtask-then-action prediction [29]) or acts on the task label alone (Run 1.4), against human-provided subtask guidance at inference, where a human oracle labels each subtask, advancing to the next one only after the prior one is judged complete. Both autonomous policies fail completely (0.0% success, at 54.9% and 54.3% progress respectively) since one misstep is enough to stall the whole rollout. With human-provided subtask guidance, Run 1.3 reaches 76.0% success against 16.0% for Run 1.4 – despite identical guidance, so subtask training at training time, not guidance at inference, is what drives the gap.

4.2.4 Observation: policies exhibit recovery and complex bimanual behaviors

FineART-VLA reproduced notable behaviors that the dataset may contain in small proportion, but the task instruction does not explicitly call for (Table 3). Self-correction: a sort tools policy redirected a trajectory already heading for the wrong bin. Failure recovery: a fallen cup was re-grasped and the placement completed, instead of the rollout stalling on the first error. Bimanual coordination: on a flower in vase rollout, the left arm steadied the vase so that the right could insert the flower. These behaviors indicate that policies are exhibiting closed-loop behavior and reacting to state feedback.

4.3 Cross-Embodiment Transfer (YAM)

Section 4.2 measures mid-trained policy performance on the in-distribution FineART embodiment. In this section, we evaluate whether the mid-trained FineART-VLA shows similar improved performance on an out-of-distribution embodiment. We fine-tune the best-performing checkpoint from Table 2a (Run 1.1, subtask training + KI, 300k steps) for an additional 5,000 steps on YAM, using five of the six ALOHA evaluation tasks – shelf, towel, vase, pegboard, and saucer – as the fine-tuning set. Sort tools and prepare breakfast are held out entirely from YAM fine-tuning, to test transfer to unseen tasks later in this section. We examine policies trained on 10, 25, 50, and 250 training episodes per task. Evaluations consist of 25 rollouts each (125 rollouts per checkpoint). The embodiment gap is substantial: Stationary ALOHA arms are mounted on opposite sides of the table facing each other, whereas YAM arms face forward towards the back of the table. We compare against three alternatives fine-tuned identically: the upstream π0.5 initialization with no FineART mid-training, and two checkpoints mid-trained on different datasets (ABC, and Ai2’s MolmoAct2 YAM release). FineART is the only one of the three datasets entirely on a different embodiment.

4.3.1 Does mid-training on FineART reduce embodiment-specific downstream data requirements?

The FineART-VLA (orange) yields a 10× reduction in downstream data compared to a checkpoint with no mid-training (black) in Fig. 6. FineART-VLA reaches 26.4% SR at 25 fine-tuning episodes per task, compared to the 18.4% SR at 250 episodes per task when there is no mid-training. Both checkpoints start from the same π0.5 weights and differ only in whether the FineART mid-training occurred, so the gain comes from the mid-training stage rather than the underlying pre-training or model architecture.

4.3.2 How does FineART-VLA performance on YAM (cross-embodiment) compare to policies from YAM open source datasets?

We compare FineART-VLA to ABC and MolmoAct2, both of which include the YAM embodiment in their pre-training dataset. On the five evaluation tasks in Fig. 6a, at the smallest budget FineART-VLA is the weakest (14.4% against 24.8% for ABC and 20.8% for MolmoAct2), which is consistent with the embodiment gap: ten episodes per task is not enough YAM data to re-target ALOHA kinematics. FineART-VLA is the only checkpoint whose success rate increases monotonically to 33.6% SR at 1250 episodes. MolmoAct2’s SR plateaus in the low twenties despite a 25-fold increase in data (20.8% to 24.8%), while ABC peaks mid-sweep at 34.4% before dropping to 24.0%. Both baselines score 0.0% success zero-shot, indicating task-embodiment fine-tuning is necessary regardless of mid-training dataset.

4.3.3 Does FineART-VLA pick the right object among distractors on YAM?

An additional task tests spatial grounding: put the cable into the bin, with a bread roll and a donut as distractors, where the target and both distractors are absent from the YAM fine-tuning data. FineART-VLA improves monotonically from 36.0% to 60.0% success as the budget grows. ABC starts higher at ten episodes (52.0%) but then drops sharply as the fine-tuning set grows, suggesting the representation is overwritten, not refined, by fine-tuning.

4.3.4 Does fine-tuning on non-relevant YAM tasks enable transfer to unseen long-horizon tasks?

The most striking result is on tasks that appear nowhere in the YAM fine-tuning data. After the same 5k-step fine-tune on the five basic tasks (Fig. 6a), we evaluate FineART-VLA on prepare breakfast, which requires placing a plate on the tray and then arranging two donuts and a slice of bread on it. As shown in Fig. 7, the policy reaches 28.0% SR and 78.3% PR with autonomous hierarchical execution, executing the full multi-stage sequence on a robot it had seen only through five unrelated tasks. It also emits the Adjust subtask to realign the tray, a behavior annotated in the FineART ALOHA data and never performed in the YAM fine-tuning set. Sort tools, also unseen on YAM, reaches 12.0% SR and 60.0% PR. These results indicate that subtask-level task structure transfers across the embodiment gap, and the fine-tuning trajectories supply the kinematics for embodiment-specific motor patterns.

5 Conclusion

To our knowledge, FineART is the largest open-source subtask-annotated manipulation dataset: 533,913 labels covering every one of its 40,543 episodes and 151 tasks, spanning hundreds of distinct object classes plus a long tail of contact-rich skills beyond ordinary pick-and-place tasks (Section 3).

Experiments with our FineART-VLA policy indicate that the FineART dataset (Section 4) enables meaningful performance gains on in-distribution and out-of-distribution tasks and across embodiments. Inclusion of long-tail tasks enables higher performance across partially in-distribution and out-of-distribution evaluation tasks, indicating object variety alone is not sufficient for task diversity. Subtask training enables FineART-VLA to parse instructions, which a flat policy cannot, and to execute long-horizon sequential tasks. In cross-embodiment experiments with minimal target embodiment fine-tuning (Stationary table-top ALOHA to Mobile YAM), our FineART-VLA policy completes long-horizon tasks not present in the target embodiment fine-tuning set. These results indicate that FineART-VLA learned subtask-level planning during ALOHA mid-training, and the target embodiment fine-tuning trains the policy on the target robot’s kinematics without forgetting the sequential task planning.

Limitations and Future Work.

Our dataset focuses on a single robot platform, and our policy is only trained for about 10% of an epoch. Training for longer and testing more base models would likely show a larger impact of dense subtask annotations. Given the impressive performance gains from our subtask-conditioned FineART-VLA policy, we are excited to see progress on subtask-conditioned world models [16], reward models [8], and advantage-conditioned RL [30], which enables learning from rejected rollouts instead of discarding them. VLM-automated labeling shows promise for reducing subtask annotation burden as the dataset grows [5]. Finally, while fine-tuning on five non-relevant YAM tasks enabled transfer to unseen, long-horizon YAM tasks, it is unclear which types of fine-tuning tasks drove cross-embodiment transfer. A task, annotation, and embodiment sweep [28, 9] would help isolate the effects of mid-training and fine-tuning dataset diversity. By releasing the FineART dataset, model, and training code, we present a canvas for future exploration in our field.

Acknowledgments

We thank our robot operators for data collection, annotation, evaluations, and logistics, and the many colleagues across engineering and program leadership who supported this work. See Appendix C for full acknowledgments.

References

• [1] M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, et al. (2022) Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. In Conf. on Robot Learning (CoRL), pp. 287–318. Cited by: §1, §1, §2.2, §2.3.
• [2] A. Allshire, H. G. Singh, R. Singh, A. Rashid, H. Choi, D. McAllister, J. Yu, Y. Chen, H. Huang, P. Abbeel, et al. (2026) Scalable Behavior Cloning with Open Data, Training, and Evaluation. Note: arXiv:2606.27375 Cited by: §1, §2.1, Table 1, §3.2.
• [3] S. Belkhale, T. Ding, T. Xiao, P. Sermanet, Q. Vuong, J. Tompson, Y. Chebotar, D. Dwibedi, and D. Sadigh (2024) RT-H: Action Hierarchies Using Language. In Robotics: Science and Systems (RSS), Cited by: §1, §2.2.
• [4] H. Bharadhwaj, J. Vakil, M. Sharma, A. Gupta, S. Tulsiani, and V. Kumar (2024) RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunking. In IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 4788–4795. Cited by: Table 1.
• [5] N. Blank, M. Reuss, M. Rühle, Ö. E. Yağmurlu, F. Wenzel, O. Mees, and R. Lioutikov (2024) Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Models. In Conf. on Robot Learning (CoRL), pp. 4158–4187. Cited by: §5.
• [6] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, et al. (2023) RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. In Conf. on Robot Learning (CoRL), pp. 2165–2183. Cited by: §1, §2.3.
• [7] R. Cadene, S. Alibert, F. Capuano, M. Aractingi, A. Zouitine, P. Kooijmans, J. Choghari, M. Russi, C. Pascal, S. Palma, M. Shukor, J. Moss, A. Soare, D. Aubakirova, Q. Lhoest, Q. Gallouédec, and T. Wolf (2026) LeRobot: An Open-Source Library for End-to-End Robot Learning. In The Fourteenth International Conference on Learning Representations, Cited by: §4.1.
• [8] Q. Chen, J. Yu, M. Schwager, P. Abbeel, Y. Shentu, and P. Wu (2026) SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulation. In Int. Conf. on Learning Representations (ICLR), Cited by: §5.
• [9] R. Doshi, H. Walke, O. Mees, S. Dasari, and S. Levine (2024) Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation. In Conf. on Robot Learning (CoRL), pp. 496–512. Cited by: §2.3, §5.
• [10] D. Driess, J. T. Springenberg, B. Ichter, L. Yu, A. Li-Bell, K. Pertsch, A. Z. Ren, H. Walke, Q. Vuong, L. X. Shi, and S. Levine (2025) Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better. Note: arXiv:2505.23705 Cited by: §2.3, §4.1.
• [11] H. Fang et al. (2026) MolmoAct2: Action Reasoning Models for Real-world Deployment. Note: arXiv:2605.02881 Cited by: §2.1, Table 1.
• [12] H.-S. Fang, H. Fang, Z. Tang, J. Liu, C. Wang, J. Wang, H. Zhu, and C. Lu (2024) RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot. In IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 653–660. Cited by: §1, §2.1, Table 1.
• [13] Z. Fu, T. Z. Zhao, and C. Finn (2024) Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation. In Conf. on Robot Learning (CoRL), pp. 4066–4083. Cited by: §1, §2.1.
• [14] Gemini Robotics Team, S. Abeyruwan, J. Ainslie, J.-B. Alayrac, M. G. Arenas, T. Armstrong, A. Balakrishna, R. Baruch, M. Bauza, M. Blokzijl, et al. (2025) Gemini Robotics: Bringing AI into the Physical World. Note: arXiv:2503.20020 Cited by: §2.3.
• [15] Gemma Team et al. (2024) Gemma 2: Improving Open Language Models at a Practical Size. Note: arXiv:2408.00118 Cited by: §4.1.
• [16] Y. Guo, L. X. Shi, J. Chen, and C. Finn (2026) Ctrl-World: A Controllable Generative World Model for Robot Manipulation. In Int. Conf. on Learning Representations (ICLR), Cited by: §5.
• [17] S. Hu and T. Horii (2026) SADP: Subgoal-Aware Diffusion Policy for Long-Horizon Manipulation Learned from Foundation Model Generated Demonstrations. Note: arXiv:2605.16871 Cited by: §2.3.
• [18] T. Jiang, T. Yuan, Y. Liu, C. Lu, J. Cui, X. Liu, S. Cheng, J. Gao, H. Xu, and H. Zhao (2025) Galaxea Open-World Dataset and G0 Dual-System VLA Model. Note: arXiv:2509.00576 Cited by: §2.3.
• [19] D. Kalashnikov, J. Varley, Y. Chebotar, B. Swanson, R. Jonschkowski, C. Finn, S. Levine, and K. Hausman (2021) Scaling Up Multi-Task Robotic Reinforcement Learning. In Conf. on Robot Learning (CoRL), pp. 557–575. Cited by: §2.1, Table 1.
• [20] A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y. Chen, K. Ellis, et al. (2024) DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset. In Robotics: Science and Systems (RSS), pp. 1–13. Cited by: §1, §2.1, Table 1, §3.2.
• [21] B. Kim, R. Wang, D. Acuna, J. Jung, A. Trevithick, B. Cui, Y. Choi, and P. Ammanabrolu (2026) How to Instruct Your Robot: Dense Language Annotations Power Robot Policy Learning. Note: arXiv:2605.17077 Cited by: §2.3.
• [22] M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn (2024) OpenVLA: An Open-Source Vision-Language-Action Model. In Conf. on Robot Learning (CoRL), pp. 2679–2713. Cited by: §1, §2.3.
• [23] R. Krishna, K. Hata, F. Ren, L. Fei-Fei, and J. C. Niebles (2017) Dense-Captioning Events in Videos. In IEEE Int. Conf. on Computer Vision (ICCV), pp. 706–715. Cited by: §2.2.
• [24] Y. Li, Y. Deng, J. Zhang, J. Jang, M. Memmel, R. Yu, C. R. Garrett, F. Ramos, D. Fox, A. Li, A. Gupta, and A. Goyal (2025) HAMSTER: Hierarchical Action Models for Open-World Robot Manipulation. In Int. Conf. on Learning Representations (ICLR), Cited by: §2.3.
• [25] C. Lynch, A. Wahid, J. Tompson, T. Ding, J. Betker, R. Baruch, T. Armstrong, and P. Florence (2023) Interactive Language: Talking to Robots in Real Time. IEEE Robotics and Automation Letters (RA-L), pp. 1–8. External Links: Document Cited by: §2.3.
• [26] O. Mees, L. Hermann, E. Rosete-Beas, and W. Burgard (2022) CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks. IEEE Robotics and Automation Letters (RA-L) 7 (3), pp. 7327–7334. Cited by: §1, §2.2.
• [27] T. Motoda, M. Murooka, R. Nakajo, M. A. Muttaqien, K. Makihara, H. Oh, K. Shirai, F. Erich, R. Hanai, and Y. Domae (2025) AIST Bimanual Manipulation Dataset. Note: https://aistairc.github.io/aist_bimanip_site/ Cited by: §2.1.
• [28] Open X-Embodiment Collaboration et al. (2024) Open X-Embodiment: Robotic Learning Datasets and RT-X Models. In IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 6892–6903. Cited by: §1, §2.1, Table 1, §5.
• [29] Physical Intelligence et al. (2025) π0.5: a Vision-Language-Action Model with Open-World Generalization. Note: arXiv:2504.16054 Cited by: §A.1, §2.3, §4.1, §4.1, §4.1, §4.1, §4.2.3, §4.
• [30] Physical Intelligence et al. (2025) π0.6∗: a VLA That Learns From Experience. Note: arXiv:2511.14759 Cited by: §5.
• [31] L. X. Shi, B. Ichter, M. Equi, L. Ke, K. Pertsch, Q. Vuong, J. Tanner, A. Walling, H. Wang, N. Fusai, A. Li-Bell, D. Driess, L. Groom, S. Levine, and C. Finn (2025) Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models. In Int. Conf. on Machine Learning (ICML), pp. 54919–54933. Cited by: §1, §2.3.
• [32] FlashRT External Links: Link Cited by: §A.6, §A.6.
• [33] H. R. Walke, K. Black, T. Z. Zhao, Q. Vuong, C. Zheng, P. Hansen-Estruch, A. W. He, V. Myers, M. J. Kim, M. Du, A. Lee, K. Fang, C. Finn, and S. Levine (2023) BridgeData V2: A Dataset for Robot Learning at Scale. In Conf. on Robot Learning (CoRL), pp. 1723–1736. Cited by: Table 1, §3.2.
• [34] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. V. Le, and D. Zhou (2022) Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems 35, pp. 24824–24837. Cited by: §2.3.
• [35] X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer (2023) Sigmoid Loss for Language Image Pre-Training. In Int. Conf. on Computer Vision (ICCV), pp. 11975–11986. Cited by: §4.1.
• [36] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn (2023) Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. In Robotics: Science and Systems (RSS), Cited by: §1, §2.1.

Appendix

Appendix A FineART-VLA: Architecture and Mid-Training Details

Figure 8 gives the full computational path behind Section 4.1’s architecture, Fig. 9 gives the attention masks behind 1’s two training shapes, and Table 5 gives the mid-training configuration behind all of our subtask/KI runs in Table 2(a).

The LM head decodes the subtask on 30% of samples (λtext=1), and the action expert conditions on it for the remaining 70% (1). The action expert always conditions on the 14-dim joint state qt, discretized into 256 bins and embedded as text in the same prompt rather than passed through the backbone. The FAST and the flow-matching output never attend to each other. With knowledge insulation on, the flow loss’s gradient never reaches θVLM while the FAST loss – computed entirely by the backbone’s own LM head – is unaffected either way (Section A.2). Because a sample either generates the subtask (30%) or predicts the FAST and action outputs conditioned on it (70%), these two cases use two different attention patterns rather than one shared sequence, shown side by side in Fig. 9 (Section A.2).

A.1 Task-Balanced Sampling

FineART’s 151 tasks range from a few dozen episodes to tens of thousands, so sampling frames uniformly from the pooled dataset would spend most of training on whichever tasks happen to be over-represented, starving all others. We use the temperature-sampling scheme from [29] behind the “task-balanced, α=0.5” row of Table 5(a): task i, with ni valid (non-idle) frames, is drawn with probability

and once a task is chosen, the frame itself is drawn uniformly from that task’s ni valid frames. α interpolates between two extremes. At α=0, Eq. 3 gives p⁡(task=i)=1/151 for every task regardless of size, so a task with a handful of episodes gets exactly as many gradient steps as one an order of magnitude larger. At α=1, it reduces to ni/∑jnj, i.e., sampling frames uniformly from the pooled dataset, so the largest tasks dominate in direct proportion to their size. We use α=0.5 for every run in this paper: a task twice as large as another is still sampled more often, but only by a factor of 2 rather than 2.

We do not implement Eq. 3 as a single per-frame weight vector handed to a weighted sampler since building and drawing from a categorical distribution over FineART’s 185.5M frames is beyond what torch.multinomial’s categorical cap (224 entries) supports. We sample hierarchically: draw a task from Eq. 3 by an inverse-CDF lookup on the 151-entry task distribution, then draw a frame uniformly from a precomputed per-task index of valid frames, costing two small lookups per sample instead of one lookup into a 185.5M-entry table.

A.2 Knowledge Insulation

Section 4.1 asserts that knowledge insulation makes ∇θVLMℒflow=0. Here we show why. At every layer l=1,…,L of the joint attention, drop the head dimension and write the VLM stream’s own query/key/value as Qvlm(l),Kvlm(l),Vvlm(l) (projections of the backbone hidden state hvlm(l−1)) and the action expert’s as Qact(l),Kact(l),Vact(l) (projections of hact(l−1), its own 300M-parameter Gemma tower). Because the prefix never attends into the suffix (Fig. 9), the VLM stream’s attention is ordinary self-attention,

and its final layer’s output realizes the backbone representation zθ of 1. The action stream’s attention additionally reads the VLM stream’s keys and values (it is conditioned on the backbone, not vice versa):

where [⋅;⋅] concatenates along the key/value sequence axis and Mvlm,Mact are the additive masks realizing Fig. 9(b). Without KI, κ(l)=Kvlm(l) and ν(l)=Vvlm(l) directly. With KI, κ(l)=sg⁡(Kvlm(l)) and ν(l)=sg⁡(Vvlm(l)), where sg⁡(⋅) is the stop-gradient operator: the identity in the forward pass (sg⁡(x)=x) but a constant under differentiation (∂sg⁡(x)/∂x≡0). Figure 10 draws this split for a single layer.

Because sg⁡(⋅) is the identity in the forward direction, Eq. 5 takes the exact same numerical value whether or not KI is enabled. KI is a purely backward-pass ablation, changing which parameters receive gradient and not what the model computes. Let θVLM denote every backbone parameter feeding Kvlm(l),Vvlm(l) at any layer, and note that ℒflow (Eq. 2) is read out of the action stream’s outputs only, ℒflow=g⁡(Attact(1),…,Attact(L)), never touching Attvlm(l) directly. With KI, for every layer l,

so by the chain rule ∇θVLMℒflow=∑l=1L∂ℒflow∂Attact(l)⋅∂Attact(l)∂θVLM=0. This split is applied uniformly at every one of the L backbone layers (once per layer of the fused forward), so no VLM parameter at any depth – not just a given layer’s own K/V projection weights, but every earlier embedding and layer that produced hvlm(l−1) – can reach ℒflow through this path either. sg⁡(⋅) severs the graph at that node regardless of what feeds into it upstream.

The FAST-tokenized output (Fig. 8) is computed from the VLM stream, not the action stream. FAST tokens are appended to the prefix’s own text tokens before layer 1 (Fig. 9(b): “FAST tokens” shares the bidirectional-prefix side with “Subtask + State”) and read out through Attvlm(L) using the same PaliGemma LM head that predicts the subtask. Both ℒtext and ℒfast are sliced from the same VLM-stream output tensor, which Eq. 5 never modifies. So

by the same argument in reverse: neither loss’s computational graph passes through an sg⁡(⋅) node, so the backbone continues to receive full gradient from both. KI insulates the backbone from the continuous action loss only, not from the discrete one – consistent with “the backbone is shaped only by the token-level losses” (Section 4.1).

In no panel does the prefix attend back into a generated or predicted suffix, and the FAST and continuous-action outputs are mutually invisible so neither loss leaks into the other. This masking pattern is fixed by the recipe regardless of knowledge insulation.

A.3 Optimization Objective

Equation 2 gives the three loss terms. In practice all of them are computed on every action chunk regardless of λtext since the same chunk is flow-matched and FAST-tokenized at once. We also put a z-loss on the text logits, weight 10−4: without it, a large-vocabulary head like PaliGemma’s 257K tokens can let its log-partition function drift while cross-entropy itself persists. The recipe stack can drop the subtask context at random during training, so the policy learns to fall back on the task string alone. We leave that dropout at 0 for every run in this paper.

Mid-training required 300K steps at a global batch of 64 – about 10% of one epoch over FineART’s 185.5M frames – roughly a week of wall-clock time on a single 8×H100 node. We ran the four subtask/KI ablations on separate nodes at the same time rather than one after another, so all four checkpoints in Table 2(a) were ready within that same week.

A.4 Inference

At deployment, the same checkpoint can be executed in three different ways, independent of how it was mid-trained (Fig. 11). All three call the same action expert and flow-matching head from Fig. 8. They differ in where the low-level conditioning comes from and how often it gets refreshed.

Chunk-level execution is the same in all three modes: a fresh action chunk is predicted from a new observation every n_action_steps control steps while the policy runs open-loop in between. In (b) and (c), the subtask is held across several chunks so that it spans seconds rather than a single 50-step chunk and matches how subtasks are actually distributed in training. We run inference eager rather than compiled because the action expert’s prompt embeds the discretized joint state as text. With token count shifting with almost every observation, a compiled graph re-specializes on nearly every call, so compilation never pays off.

A.5 Language Runtime and Subtask Execution

FineART-VLA integrates with LeRobot’s interactive rollout runtime, which separates language-level instruction updates from action-chunk execution. The runtime keeps the policy, observation processors, and robot connection initialized throughout a session. A terminal interface and a programmatic controller expose the same instruction-update and execution operations.

Human-provided subtasks.

An operator can replace the active instruction during execution without reloading the model. Under synchronous inference, the runtime discards queued policy actions computed under the previous instruction, and the next prediction uses the updated instruction. When real-time chunking is enabled, the new instruction conditions the next action chunk, which is blended with the remaining action prefix.

Autonomous subtask inference.

Given a high-level goal, the runtime periodically requests a subtask from FineART-VLA using the current observation. The generation prompt is constructed from the checkpoint’s saved training recipe, preserving the prompt structure used for subtask supervision. The generated text becomes the active instruction for subsequent action predictions and remains in effect until replaced. Each planning query receives the original goal. The replanning interval is configurable and measured from the application of the preceding subtask; the runtime does not itself detect subtask completion.

A.6 Efficient Training and Inference

Training.

The implementation shares backbone computation across language, FAST-token, and flow-matching objectives when their corresponding supervision is present. To amortize visual-language prefix computation, multiple independent noise and timestep draws reuse the same prefix, with five draws by default. Action embeddings and output projections are vectorized across these draws. Attention masks isolate the action sequences and prevent access to FAST action targets, and the flow losses are averaged across draws.

Additional optimizations include sparse and bucketed cross-entropy computation, per-layer vision activation checkpointing, and fused AdamW updates. Optional backends support compiled cross-entropy and FlashRT [32] adaptive RMSNorm kernels. For flow-only batches with knowledge insulation and no language or FAST supervision, the implementation can also omit the backbone’s backward graph.

Inference.

Autoregressive subtask generation uses incremental key–value caching. The action-denoising implementation reuses the visual-language prefix cache across steps and crops appended action entries instead of copying the cache at each step.

An optional FlashRT backend provides calibrated FP8 MLP kernels for Gemma and SigLIP. These kernels are inference-only, change numerical precision, and are disabled by default.

Liang Su, author of FlashRT [32], contributed substantial implementation and optimization work underlying this section, including denoising-cache improvements, training-path optimizations, and FlashRT kernel integration.

Appendix B Evaluations

Our evaluations total 3,400 rollouts across 23 checkpoints between experiments on ALOHA and YAM, summarized in Table 6.

B.1 Procedure

Evaluations of our scale demand procedural rigor. Across benchmark suites, we predetermine 25 initial conditions, namely position and orientation, for each task (Table 7).

Fig. 12 illustrates the 25 initial conditions for an in-distribution benchmark task: put cup on shelf.

B.2 Rubric

We evaluate each checkpoint by progress rate based on subtask completion percentage. Subtasks are defined such that one subtask must be completed before the next can begin (Table 8). Prepare breakfast is an exception, with no prescribed order after its first subtask.

Appendix C Acknowledgments

Our work would not be possible without the support of colleagues and friends at Scale AI, Hugging Face, and the community.

• •

  We thank Alex Finch, Angel Uribe, Luke Pulaski, and Marci Ramos for their work in operations for data collection; Ana Paula Estevez, Juan Carlos Becerril, Jose Antonio Gonzalez, Paulina Vergara Montoya, and Astrid Hernandez Galvez for their work in operations for data annotation; Aidan Walker, Amanda Dee, Angel Uribe, Devender Bankoti, Luke Pulaski, and Marci Ramos for their work in operations for evaluation; and Aidan Walker, Aliyah Dela Cruz, Amanda Dee, Angel Uribe, Caroline Doubane, Devender Bankoti, Isaiah Smith, Joshua Diaz, Katerina Connearney, Marci Ramos, Reeya Shrestha, Rudy Garcia, Sherwood Yee, Tony Alfatooni, Travis Bringas, and Veronika Tsvelodub for their work as on-site contributors.

• •

  We thank Selam Gano, Sidney Nimako, Tyler Smithline, Martin Lombardo, Matias Del Carlo, Fran Espeche, Gabriel Blanchard, Shreyas Chakravarthula, Eric Taylor, and Tim Lu for their work in hardware engineering for data collection; Martin Lombardo, Matias Del Carlo, Fran Espeche, and Gianluca Bobbio for their work in software engineering for data collection; Gregorio Freidin, Matias Carou, Santiago Illi, and Gianluca Bobbio for their work in software engineering for data processing; and Garrett Matsuda, Juan Cabrera, Malena Goñi, Conrado Mader Blanco, Ezequiel Romio, Micaela Alvarez, Luis Lopez Castaneda, Ankit Vedak, Gianluca Bobbio, and Tim Lu for their work in software engineering for data annotation.

• •

  We thank Caroline Clark, Kendyl Burkitt, and Molly Taudvin for their go-to-market support; and Javier Gonzalez for their cross-functional support.

• •

  We thank Harsha Mohan, Michel Aractingi, Caroline Pascal, Carlos Jerez, Ke Wang, and Khalil Meftah for their early discussions on policy and dataset design.

• •

  We thank Liang Su for their work on training and inference speed optimizations for FineART-VLA; and Steven Palma and the LeRobot team for their contributions to the interactive rollout runtime.

• •

  We thank Ben Levin, Natasha Dadabhoy, and Joe Fox Jr. for their leadership and for approving the public release of the dataset.