H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning
H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning
This research presents H-JEPA, an end-to-end method for training hierarchical action-conditioned JEPAs. The approach optimizes progress toward goals at a top level while using lower levels to predict finer details, effectively discarding fast, unpredictable data in favor of slower, task-relevant states. Experiments across four simulated environments show that a three-level hierarchy improves success rates on Visual AntMaze from 18% to 73% with reduced compute. The method also extends to real-robot videos from DROID using inverse-dynamics supervision, improving offline planning fidelity while lowering planner compute requirements.
The paper introduces H-JEPA, a hierarchical world model architecture that decomposes planning across timescales to improve long-horizon navigation and manipulation in simulation and on real-robot video data.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
s over a flat JEPA; on Visual AntMaze, a three-level hierarchy raises success from 18% to 73% using less planner compute. Ablations attribute these gains to both temporal decomposition and higher-level goal representations. With inverse-dynamics supervision, the approach extends to diverse real-robot videos from DROID, where hierarchy improves offline planning fidelity at lower planner compute.
Open source S4
H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning
Open source S1
arXiv:2610.06805v2 Announce Type: replace-cross Abstract: Long-horizon planning with latent world models requires reasoning across timescales and levels of abstraction. Existing task-agnostic JEPA world models predict and plan in a single latent space, often at a single timescale. We introduce H-JEPA, an end-to-end recipe for training a hierarchy of action-conditioned JEPAs in which each level predicts farther ahead
Open source S2
in its own learned latent space. Planning proceeds top-down: the top level optimizes progress toward the goal, and each level's predictions become subgoals for the planner below it. When factors in the data evolve at separated timescales, higher levels discard fast, unpredictable detail and retain slower task-relevant state. Across four simulated navigation and manipulation environments, hierarchical planning improve
Open source S3
Source：arXiv Robotics — research abstracts · arxiv.org