README.md

---
license: mit
base_model: quentinll/lewm-pusht
library_name: jepa.cpp
pipeline_tag: image-feature-extraction
tags:
  - jepa
  - ggml
  - gguf
  - jepa.cpp
  - world-model
  - robotics
  - image-feature-extraction
---

# LeWorldModel Push-T — GGUF for jepa.cpp

**LeWorldModel** Push-T: a ViT-Ti/14 image encoder plus a latent, action-conditioned world-model
predictor, converted to GGUF for [jepa.cpp](https://github.com/aselimc/jepa.cpp) — a ggml C/C++ engine that runs it on a plain CPU
with no Python and no PyTorch. The encoder's CLS token is projected to a world-model state, and six
AdaLN-zero causal blocks conditioned on a 10-d action roll that state forward.

**18 M** parameters; D = 192, 12 layers, 3 heads, patch 14, 224x224. Everything the engine needs — dimensions, positional scheme,
preprocessing recipe, and class labels where there are any — travels inside the file, so inference needs
one binary and one GGUF and nothing else.

## Run it

```bash
git clone --recursive https://github.com/aselimc/jepa.cpp && cd jepa.cpp
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release && cmake --build build -j
hf download jepacpp/lewm-pusht-GGUF lewm-pusht-f16.gguf --local-dir models/gguf

# one image -> a world-model state, then 8 random action steps
build/jepa-embed      -m lewm-pusht-f16.gguf -i photo.jpg --pool lewm -t 32
build/jepa-worldmodel -m lewm-pusht-f16.gguf --image photo.jpg --random-actions 8 -t 32
```

`--pool` selects `mean`, `cls`, `lewm` or `none` (the full token map); `-o` writes a `.npy`.
`scripts/download_models.sh` fetches whole sets at once. The C API is one header,
[`include/jepa.h`](https://github.com/aselimc/jepa.cpp/blob/main/include/jepa.h) — full reference on the [C API page](https://aselimc.github.io/jepa.cpp/api/).

## Files

| file | size | sha256 (first 16) | tier | measured against the PyTorch reference |
|---|---|---|---|---|
| `lewm-pusht-f32.gguf` | 68.8 MiB | `30cc97fdbde7129f` | exact | cos mean 1.000000, worst 1.000000, `rel_max` 1.0e-06 ᵖ |
| `lewm-pusht-f16.gguf` | 37.7 MiB | `24f0a647b4ce1385` | parity | cos mean 1.000000, worst 1.000000 ᵖ |
| `lewm-pusht-q8_0.gguf` | 23.1 MiB | `3cadd1cb7d57ecd8` | parity | cos mean 0.999913, worst 0.999895 ᵖ |
| `lewm-pusht-q4_0.gguf` | 15.3 MiB | `f29b58b62455a7a1` | advisory | cos mean 0.998547, worst 0.995504, `cls` 0.998194, `emb` 0.992647, `pred_next` 0.991251 ᵈ |
| `lewm-pusht-q4_k.gguf` | 15.3 MiB | `0c03dc7c38181d56` | advisory | cos mean 0.998779, worst 0.996803, `cls` 0.998297, `emb` 0.993206, `pred_next` 0.991625 ᵈ |

<sub>ᵖ `tests/test-parity` on the CPU backend, stored reference input, 32 threads, worst sample — [docs/parity.md](https://aselimc.github.io/jepa.cpp/parity/). ᵈ `scripts/gguf_dequant_selftest.py`: the dequantized weights through the numpy reference graph at f32 activations, so the figure is the weight error alone — [docs/quantization.md](https://aselimc.github.io/jepa.cpp/quantization/). `cos mean` is the mean per-token cosine of `last_hidden_state`, `worst` its single worst token.</sub>

**Tiers.** `exact` — reproduces the PyTorch reference to the printed precision on the CPU. `parity` —
passes its family's `test-parity` thresholds. `advisory` — below 8 bits per weight, which is not a parity
configuration: the results are reported, only the derived tensors and the top-1 are gated. Which file to
ship: [Accuracy → which dtype](https://aselimc.github.io/jepa.cpp/accuracy/#which-dtype-to-ship).

Full checksums:

```
30cc97fdbde7129f1f545a670253c0a0630308cec0645550ec181987bc0f210e  lewm-pusht-f32.gguf
24f0a647b4ce1385ac9e52a596db07befd5f67f4f175fd68c615c03447d6ccd3  lewm-pusht-f16.gguf
3cadd1cb7d57ecd8078776e4b414ff9a2c48ba28e26f9127038306268a042c38  lewm-pusht-q8_0.gguf
f29b58b62455a7a1197596112941f5518b819376af052ee0c38473c214757a11  lewm-pusht-q4_0.gguf
0c03dc7c38181d56f26efd8ce960414f675d675f845ee7df7353ac373010c6fd  lewm-pusht-q4_k.gguf
```

Verify a download with `sha256sum -c`. The other types `jepa-quantize` can produce (`q4_1`, `q5_0`,
`q5_1`, `q5_k`, `q6_k`, measured in [quantization](https://aselimc.github.io/jepa.cpp/quantization/)) are not published here; make
them locally with `build/jepa-quantize lewm-pusht-f16.gguf out.gguf q6_k -t 32`.

## Measured

Every figure below is read from a committed artifact of jepa.cpp [`00bfd4e`](https://github.com/aselimc/jepa.cpp/commit/00bfd4e) by `scripts/hf_publish.py` — [parity](https://aselimc.github.io/jepa.cpp/parity/), [quantization](https://aselimc.github.io/jepa.cpp/quantization/), [accuracy](https://aselimc.github.io/jepa.cpp/accuracy/), [performance](https://aselimc.github.io/jepa.cpp/performance/) and `tests/results/*.json`.

**Imagenette k-NN** — 1000 queries against a gallery of 2000 (`train2000`), the frozen `emb` feature, k = 20 cosine vote. Nothing is trained.

| backend | dtype | k-NN top-1 % | centroid top-1 % | agreement with PyTorch % | mean feature cosine |
|---|---|---:|---:|---:|---:|
| pytorch | f32 | **27.00** | 24.10 | — | 1 |
| jepa.cpp | f32 | 26.70 | 24.00 | 97.40 | 0.999994 |
| jepa.cpp | f16 | 26.60 | 24.00 | 97.40 | 0.999994 |
| jepa.cpp | q8_0 | 26.70 | 24.00 | 94.40 | 0.999929 |
| jepa.cpp | q4_k | 28.20 | 24.60 | 83.40 | 0.994496 |

**Speed** — the encoder graph at f16 on 32 threads (AMD Ryzen Threadripper PRO 7995WX 96-Cores): 10 ms per image against PyTorch's 17 ms. The same shape on NVIDIA RTX 4500 Ada Generation: 0.9 ms. Peak RSS at f16: 47 MiB.

The most quantization-robust model in the set — narrow, shallow, with BatchNorm folded into the MLPs —
and the only one whose predictor `tests/test-predictor` gates with a bit-exactness causality check:
perturbing step *t* leaves every prediction before *t* bit-identical.

## Source, licence and attribution

Converted from [`quentinll/lewm-pusht`](https://huggingface.co/quentinll/lewm-pusht).

**MIT.** The source checkpoint is published by **quentinll** for the `le-wm` project
([github.com/lucas-maes/le-wm](https://github.com/lucas-maes/le-wm), MIT); `license: mit` on the model
card. These GGUF files are the same weights re-serialised into the GGUF container, quantized where the
file name says so, and carry the MIT notice. Cite the
[LeWorldModel paper](https://arxiv.org/abs/2603.19312).

The licence travels inside every GGUF as `general.license` and the origin as `general.source_url`;
`build/jepa-info <file> --kv` prints them. jepa.cpp's own code is MIT.

## Conversion

Produced by jepa.cpp [`00bfd4e`](https://github.com/aselimc/jepa.cpp/commit/00bfd4e):

```bash
scripts/download_models.sh --convert lewm
python scripts/convert.py --family lewm --src models/quentinll/lewm-pusht --ftype f16
#   ... and again with --ftype f32 for the f32 file

for q in q8_0 q4_0 q4_k; do
  build/jepa-quantize models/gguf/lewm-pusht-f32.gguf \
      models/gguf/lewm-pusht-$q.gguf $q -t 32
done
```

`jepa-quantize` re-types only the 2-D attention / FFN / projection / classifier matrices; patch
embeddings, position tables, norms and biases keep the source type. The rules are in
[docs/gguf-schema.md](https://aselimc.github.io/jepa.cpp/gguf-schema/).

## Links

- Code: <https://github.com/aselimc/jepa.cpp>
- Documentation: <https://aselimc.github.io/jepa.cpp/>
- Parity fixtures: <https://huggingface.co/datasets/jepacpp/jepa.cpp-fixtures>
- All jepa.cpp GGUFs: <https://huggingface.co/jepacpp>