0xSero/GLM-5.3-569B-W4A16

Support this work → · X · GitHub · REAP paper · Cerebras REAP

GLM-5.3-569B — REAP keep-192 (W4A16 / INT4)

GLM-5.3 with 25% of its experts removed, at INT4 — 372 GB, built to serve on 4× H200 (Hopper) in vLLM or SGLang.

What this is

GLM-5.3 is a 753B mixture-of-experts model: each token uses 8 of 256 expert sub-networks per layer (~40B active). REAP (Router-weighted Expert Activation Pruning) scores each expert’s real contribution and deletes the least useful ones — no retraining. This cut keeps 192 of 256 experts per layer.

The experts are then INT4 W4A16 (compressed-tensors, AWQ), taken from the cyankiwi/GLM-5.3-AWQ-INT4 base. Only the routed experts are 4-bit; attention, the shared expert, the dense layers, and the head stay BF16. vLLM auto-selects the Marlin MoE kernel.

Which one, and what about Blackwell?

Variant	Experts kept	Size	KL vs BF16
504B	168 / 256	328 GB	0.506
569B (this)	192 / 256	372 GB	0.357

This W4A16 build is the Hopper-servable sibling of the EXL3 series. On Blackwell or consumer GPUs use the EXL3 equivalent instead — GLM-5.3-569B-EXL3-3.0bpw, which measures the same fidelity (0.361).

How close to the original is it?

KL divergence vs full BF16: 0.357 nats (sealed 25-prompt panel, full 154k vocabulary). KL is the standard “how differently do these two models predict” score — 0 = identical, lower = closer. Notably this is essentially the same as the EXL3 cut at the same expert count (0.361): the pruning drives the fidelity, the quant format barely moves it. Full numbers: fidelity study.

Why these experts

Instead of keeping the globally most-frequent experts (which deletes a domain’s specialists), each expert is scored by its largest share of any single domain’s routed work, so every domain — code, rare languages, structured output — keeps its specialists. Head-to-head vs frequency pruning: fidelity study.

Serving (vLLM, 4× H200)

vllm serve 0xSero/GLM-5.3-569B-W4A16 --tensor-parallel-size 4 --trust-remote-code --max-model-len 131072

Credits

• Z.AI / zai-org — GLM-5.3, the base model.
• cyankiwi — the GLM-5.3-AWQ-INT4 W4A16 base this prune is built on.
• Cerebras Research — REAP (arXiv:2510.13999).

Observations: glm-5.3-reap-observations-v1 · Fidelity study: glm-5.3-reap-fidelity-study · Built on 8× NVIDIA RTX PRO 6000 Blackwell.

License

Inherits the GLM-5.3 license.