0xSero/GLM-5.3-EXL3-3.0bpw

Support this work → · X · GitHub · REAP paper · Cerebras REAP

GLM-5.3 — EXL3 3.0bpw (unpruned base)

The full GLM-5.3 (all 256 experts) quantized to ~3-bit — 293 GB. This is the top of the series and the reference every pruned cut is measured against.

What this is

GLM-5.3 is a 753B mixture-of-experts model: each token is routed to 8 of 256 expert sub-networks per layer, so only ~40B parameters are active at once. This repo is that full model quantized to EXL3 3.0 bpw (turboderp’s exllamav3 trellis format) — no experts removed. Attention, the shared expert, and the router are kept at full-precision BF16; only the expert weights are compressed.

Which size should I pick?

Every row is the same model at a different quality/size trade-off — fewer experts is smaller and faster but further from the original:

Variant	Experts kept	Size	KL vs BF16 (lower = closer)
base (unpruned) (this)	256 (all)	293 GB	0.089
661B	224 / 256	258 GB	0.195
615B	208 / 256	240 GB	0.283
569B	192 / 256	223 GB	0.361
533B	180 / 256	210 GB	0.428
500B	168 / 256	197 GB	0.511

This is the reference cut. Any smaller variant below trades a little fidelity for a lot less VRAM.

How close to the original is it?

KL divergence vs full BF16: 0.089 nats, measured token-by-token over a sealed 25-prompt panel across the full 154k-token vocabulary. KL divergence is the standard “how differently do these two models predict” score: 0 = identical, lower = closer. For scale: the unpruned base scores 0.089, and this cut still reproduces the full model’s single most-likely next token about 91% of the time.

All cuts are measured the same way in the fidelity study.

Why these experts (and not the popular ones)

Naive pruning keeps the experts that fire most often — which quietly deletes the specialists that a single domain (code, a rare language, structured output) leans on. This series instead scores each expert by its largest share of any one domain’s routed work (“max-over-domain”), so every domain keeps its specialists even when they’re globally rare. On the same panel this beats frequency-based pruning by a wide margin — the head-to-head is in the fidelity study.

Running it

pip install exllamav3        # or run via TabbyAPI
# download this repo, then tensor-split it across your GPUs in TabbyAPI/exllamav3

Built for Blackwell / any CUDA GPUs with enough combined VRAM (4x 96 GB).

Credits

• Z.AI / zai-org — GLM-5.3, the base model.
• Cerebras Research — REAP (arXiv:2510.13999).
• turboderp / exllamav3 — the EXL3 format and runtime.

Observations: glm-5.3-reap-observations-v1 · Fidelity study: glm-5.3-reap-fidelity-study · Built on 8× NVIDIA RTX PRO 6000 Blackwell.

License

Inherits the GLM-5.3 license.