Add model card: contents, download, usage, citation
Browse files
README.md
ADDED
|
@@ -0,0 +1,78 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
pipeline_tag: robotics
|
| 5 |
+
tags:
|
| 6 |
+
- world-models
|
| 7 |
+
- jepa
|
| 8 |
+
- planning
|
| 9 |
+
- robotics
|
| 10 |
+
datasets:
|
| 11 |
+
- jepa-world-models/h-jepa
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# H-JEPA checkpoints
|
| 15 |
+
|
| 16 |
+
[](https://arxiv.org/abs/2610.06805)
|
| 17 |
+
[](https://h-jepa.com/)
|
| 18 |
+
[](https://github.com/kevinghst/H-JEPA)
|
| 19 |
+
[](https://huggingface.co/datasets/jepa-world-models/h-jepa)
|
| 20 |
+
|
| 21 |
+
Trained models of **H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning**
|
| 22 |
+
(Wancong Zhang\*, Basile Terver\*, Mike Rabbat, Yann LeCun†, Randall Balestriero†). The
|
| 23 |
+
[code release](https://github.com/kevinghst/H-JEPA) trains, evaluates and reproduces the paper's
|
| 24 |
+
planning results with them.
|
| 25 |
+
|
| 26 |
+
## Contents
|
| 27 |
+
|
| 28 |
+
| Folder | Models | Seeds | Checkpoints | Size |
|
| 29 |
+
|---|---|---|---|---|
|
| 30 |
+
| `ant/` (Visual AntMaze) | LeWM, H-JEPA 2/3/4 levels, HWM 2/3/4 levels | 42, 43, 44 | 21 | 2.6 GB |
|
| 31 |
+
| `fourroom/` (FourRoom Distractors) | same 7 models | 42, 43, 44 | 21 | 1.7 GB |
|
| 32 |
+
| `cube/` (OGBench Cube) | same 7 models | 42, 43, 44 | 21 | 2.7 GB |
|
| 33 |
+
| `pusht/` (Push-T) | same 7 models | 42, 43, 44 | 21 | 2.7 GB |
|
| 34 |
+
| `droid/` (DROID, 5 fps) | LeWM + IDM, HWM 2 levels, H-JEPA 2 levels | 1, 1000, 10000 | 9 | 2.7 GB |
|
| 35 |
+
|
| 36 |
+
Each run is a folder `<env>/<env>_<model>/seed<seed>/` with `<env>_<model>_object.ckpt` (the trained
|
| 37 |
+
model) and `normalizer.pt` (action/proprio normalization), the layout the training code writes.
|
| 38 |
+
`<model>` is `lewm`, `hjepa_l<n>` or `hwm_l<n>`, with `<n>` the number of levels. The DROID
|
| 39 |
+
checkpoints are the epoch-100 models.
|
| 40 |
+
|
| 41 |
+
## Download
|
| 42 |
+
|
| 43 |
+
Into `$HJEPA_HOME/ckpts`, where the code reads checkpoints:
|
| 44 |
+
|
| 45 |
+
```bash
|
| 46 |
+
# the 4 simulation environments (84 checkpoints)
|
| 47 |
+
hf download jepa-world-models/h-jepa --local-dir $HJEPA_HOME/ckpts --exclude "droid/*"
|
| 48 |
+
# DROID (9 checkpoints)
|
| 49 |
+
hf download jepa-world-models/h-jepa --local-dir $HJEPA_HOME/ckpts --include "droid/*"
|
| 50 |
+
# one environment or one seed: --include "cube/*", --include "*/seed42/*"
|
| 51 |
+
```
|
| 52 |
+
|
| 53 |
+
## Usage
|
| 54 |
+
|
| 55 |
+
The `_object.ckpt` files are pickled `HJEPA` modules, so loading them needs the code release on the
|
| 56 |
+
Python path (run from its `h_jepa/` directory):
|
| 57 |
+
|
| 58 |
+
```python
|
| 59 |
+
import torch
|
| 60 |
+
|
| 61 |
+
model = torch.load("cube/cube_hjepa_l3/seed42/cube_hjepa_l3_object.ckpt", weights_only=False)
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
To reproduce the paper's planning results, pass a checkpoint as `policy=` to `eval.py` or run the
|
| 65 |
+
evaluation scripts, as described in the code release README (simulation: section 4, DROID: section 5).
|
| 66 |
+
The evaluation tasks and datasets are in
|
| 67 |
+
[`jepa-world-models/h-jepa`](https://huggingface.co/datasets/jepa-world-models/h-jepa) (datasets).
|
| 68 |
+
|
| 69 |
+
## Citation
|
| 70 |
+
|
| 71 |
+
```bibtex
|
| 72 |
+
@article{zhang2026hjepa,
|
| 73 |
+
title = {{H-JEPA}: End-to-End Learning of Hierarchical World Models for Visual Planning},
|
| 74 |
+
author = {Zhang, Wancong and Terver, Basile and Rabbat, Mike and LeCun, Yann and Balestriero, Randall},
|
| 75 |
+
journal = {arXiv preprint arXiv:2610.06805},
|
| 76 |
+
year = {2026}
|
| 77 |
+
}
|
| 78 |
+
```
|