Ego Guidance: Enhancing Robot Policy Learning with Action-Free Human Egocentric Videos

This paper investigates the use of human egocentric videos without action supervision to improve robot policy learning. The authors propose Shared Prediction on Action and Representation (SPAR), an architecture that uses future visual state prediction alongside action prediction within a pretrained Vision-Language Model framework. Results indicate that representations learned from these videos enhance performance on out-of-distribution robot tasks, with improvements scaling with video data volume. The study also analyzes the impact of objective choices and data curation strategies for leveraging action-free videos in scalable robot learning.

Generated from attributed reports · Updated 4 hours ago

There is not enough continuous observation data to show a trend.