Kinematic Hypothesis Context: Embodiment-Conditioned Learned Video Coding for Bandwidth-Limited Robot Policy Uplinks
Kinematic Hypothesis Context: Embodiment-Conditioned Learned Video Coding for Bandwidth-Limited Robot Policy Uplinks
This research addresses the bandwidth bottleneck of camera uplinks when robots run large vision-language-action policies on remote servers. The authors introduce a 'kinematic hypothesis context,' a zero-rate conditioning signal derived from joint states, robot models, and camera calibration. By incorporating ego-motion homographies, per-link reprojections, and background memory warps into a learned video codec, the method reduces redundant bit usage for predictable robot motion. Experimental results on the LIBERO benchmark show a 23.9% reduction in bit rate compared to history-conditioned codecs and 40-56% over standard codecs like x264 and SVT-AV1. In closed-loop tests with OpenVLA-OFT, success rates nearly doubled from 22.8% to 43.1% at 6 kbps under packet loss conditions, outperforming DCVC-RT by 23.6 points. The approach generalizes across different codec hosts and real robot data.
The paper introduces a kinematic hypothesis context method that conditions video coding on robot joint states and models to reduce uplink bandwidth for remote policy execution, demonstrating significant bit-rate savings and success rate improvements in closed-loop benchmarks under packet loss.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
context saves 23.9% bit rate over a capacity-matched history-conditioned codec and 40-56% over x264, x265 and SVT-AV1, and needs 23.2% fewer bits than DCVC-RT at equal policy action deviation; trained to lower rates, it matches DCVC-RT in PSNR. In closed loop at equal rate, OpenVLA-OFT success nearly doubles from 22.8% to 43.1% at 6 kbps (72.6% with lower-rate training, 23.6 points above DCVC-RT), pi_0.5 success ris
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org