Robotics
PyTorch
world-models
jepa
planning
Basile-Terv commited on
Commit
6844abd
·
verified ·
1 Parent(s): 99b1ecb

Add model card: contents, download, usage, citation

Browse files
Files changed (1) hide show
  1. README.md +78 -0
README.md ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: pytorch
4
+ pipeline_tag: robotics
5
+ tags:
6
+ - world-models
7
+ - jepa
8
+ - planning
9
+ - robotics
10
+ datasets:
11
+ - jepa-world-models/h-jepa
12
+ ---
13
+
14
+ # H-JEPA checkpoints
15
+
16
+ [![arXiv](https://img.shields.io/badge/arXiv-2610.06805-b5212f?logo=arxiv)](https://arxiv.org/abs/2610.06805)
17
+ [![Website](https://img.shields.io/badge/Website-h--jepa.com-blue)](https://h-jepa.com/)
18
+ [![Code](https://img.shields.io/badge/GitHub-kevinghst/H--JEPA-black?logo=github)](https://github.com/kevinghst/H-JEPA)
19
+ [![Dataset](https://img.shields.io/badge/🤗%20Dataset-jepa--world--models/h--jepa-ffc107)](https://huggingface.co/datasets/jepa-world-models/h-jepa)
20
+
21
+ Trained models of **H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning**
22
+ (Wancong Zhang\*, Basile Terver\*, Mike Rabbat, Yann LeCun†, Randall Balestriero†). The
23
+ [code release](https://github.com/kevinghst/H-JEPA) trains, evaluates and reproduces the paper's
24
+ planning results with them.
25
+
26
+ ## Contents
27
+
28
+ | Folder | Models | Seeds | Checkpoints | Size |
29
+ |---|---|---|---|---|
30
+ | `ant/` (Visual AntMaze) | LeWM, H-JEPA 2/3/4 levels, HWM 2/3/4 levels | 42, 43, 44 | 21 | 2.6 GB |
31
+ | `fourroom/` (FourRoom Distractors) | same 7 models | 42, 43, 44 | 21 | 1.7 GB |
32
+ | `cube/` (OGBench Cube) | same 7 models | 42, 43, 44 | 21 | 2.7 GB |
33
+ | `pusht/` (Push-T) | same 7 models | 42, 43, 44 | 21 | 2.7 GB |
34
+ | `droid/` (DROID, 5 fps) | LeWM + IDM, HWM 2 levels, H-JEPA 2 levels | 1, 1000, 10000 | 9 | 2.7 GB |
35
+
36
+ Each run is a folder `<env>/<env>_<model>/seed<seed>/` with `<env>_<model>_object.ckpt` (the trained
37
+ model) and `normalizer.pt` (action/proprio normalization), the layout the training code writes.
38
+ `<model>` is `lewm`, `hjepa_l<n>` or `hwm_l<n>`, with `<n>` the number of levels. The DROID
39
+ checkpoints are the epoch-100 models.
40
+
41
+ ## Download
42
+
43
+ Into `$HJEPA_HOME/ckpts`, where the code reads checkpoints:
44
+
45
+ ```bash
46
+ # the 4 simulation environments (84 checkpoints)
47
+ hf download jepa-world-models/h-jepa --local-dir $HJEPA_HOME/ckpts --exclude "droid/*"
48
+ # DROID (9 checkpoints)
49
+ hf download jepa-world-models/h-jepa --local-dir $HJEPA_HOME/ckpts --include "droid/*"
50
+ # one environment or one seed: --include "cube/*", --include "*/seed42/*"
51
+ ```
52
+
53
+ ## Usage
54
+
55
+ The `_object.ckpt` files are pickled `HJEPA` modules, so loading them needs the code release on the
56
+ Python path (run from its `h_jepa/` directory):
57
+
58
+ ```python
59
+ import torch
60
+
61
+ model = torch.load("cube/cube_hjepa_l3/seed42/cube_hjepa_l3_object.ckpt", weights_only=False)
62
+ ```
63
+
64
+ To reproduce the paper's planning results, pass a checkpoint as `policy=` to `eval.py` or run the
65
+ evaluation scripts, as described in the code release README (simulation: section 4, DROID: section 5).
66
+ The evaluation tasks and datasets are in
67
+ [`jepa-world-models/h-jepa`](https://huggingface.co/datasets/jepa-world-models/h-jepa) (datasets).
68
+
69
+ ## Citation
70
+
71
+ ```bibtex
72
+ @article{zhang2026hjepa,
73
+ title = {{H-JEPA}: End-to-End Learning of Hierarchical World Models for Visual Planning},
74
+ author = {Zhang, Wancong and Terver, Basile and Rabbat, Mike and LeCun, Yann and Balestriero, Randall},
75
+ journal = {arXiv preprint arXiv:2610.06805},
76
+ year = {2026}
77
+ }
78
+ ```