Brain-Aligned Latent Actions: Imposing Premotor Cortex Geometry on a Robot's Action Vocabulary
Overview
This research paper introduces a method to align a robot's discrete latent action vocabulary with the representational geometry of the macaque ventral premotor cortex (PMv). By training a tokenizer alongside a straight-through similarity loss using PMv data recorded during shape drawing, the authors achieve a representational similarity of 0.52 on five unseen shape categories. This outperforms released tokenizers (0.20-0.26) and random kinematic encoders (0.42 at the 95th percentile). However, the study notes that kinematic descriptors like initial stroke direction match PMv better (0.65), and a label-shuffled control scores 0.41. The work is validated on a CALVIN pipeline reproducing Moto's published results.
Generated from attributed reports · Updated 9 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
No current evidence-backed claims. Missing information remains not reported.
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalBrain-Aligned Latent Actions: Imposing Premotor Cortex Geometry on a Robot's Action Vocabulary
This research paper introduces a method to align a robot's discrete latent action vocabulary with the representational geometry of the macaque ventral premotor cortex (PMv). By training a tokenizer alongside a straight-through similarity loss using PMv data recorded during shape drawing, the authors achieve a representational similarity of 0.52 on five unseen shape categories. This outperforms released tokenizers (0.20-0.26) and random kinematic encoders (0.42 at the 95th percentile). However, the study notes that kinematic descriptors like initial stroke direction match PMv better (0.65), and a label-shuffled control scores 0.41. The work is validated on a CALVIN pipeline reproducing Moto's published results.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).