0xSero commited on
Commit
cf6d850
·
verified ·
1 Parent(s): 9f39439

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md CHANGED
@@ -11,6 +11,9 @@ tags:
11
  - expert-pruning
12
  ---
13
 
 
 
 
14
  # GLM-5.3-569B — REAP keep-192 (EXL3 3.0bpw)
15
 
16
  > A **25%-expert-pruned GLM-5.3** (192 / 256 routed experts per layer, ~569B params), quantized to
@@ -44,3 +47,27 @@ agreement 0.822 vs 0.775, KLD cut by a third.
44
  | [`GLM-5.3-500B-EXL3-3.0bpw`](https://huggingface.co/0xSero/GLM-5.3-500B-EXL3-3.0bpw) | 168 | 197 GB | 0.511 |
45
 
46
  Method and the full criterion study: [`0xSero/glm-5.3-reap-fidelity-study`](https://huggingface.co/datasets/0xSero/glm-5.3-reap-fidelity-study).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  - expert-pruning
12
  ---
13
 
14
+ > [!TIP]
15
+ > **[Support this work →](https://donate.sybilsolutions.ai)** · [X](https://x.com/0xsero) · [GitHub](https://github.com/0xsero) · [REAP paper](https://arxiv.org/abs/2510.13999) · [Cerebras REAP](https://huggingface.co/collections/cerebras/cerebras-reap)
16
+
17
  # GLM-5.3-569B — REAP keep-192 (EXL3 3.0bpw)
18
 
19
  > A **25%-expert-pruned GLM-5.3** (192 / 256 routed experts per layer, ~569B params), quantized to
 
47
  | [`GLM-5.3-500B-EXL3-3.0bpw`](https://huggingface.co/0xSero/GLM-5.3-500B-EXL3-3.0bpw) | 168 | 197 GB | 0.511 |
48
 
49
  Method and the full criterion study: [`0xSero/glm-5.3-reap-fidelity-study`](https://huggingface.co/datasets/0xSero/glm-5.3-reap-fidelity-study).
50
+
51
+ ---
52
+
53
+ ## Acknowledgements
54
+
55
+ - **[Z.AI / zai-org](https://huggingface.co/zai-org)** for [GLM-5.3](https://huggingface.co/zai-org/GLM-5.3), the base model.
56
+ - **[Cerebras Research](https://github.com/CerebrasResearch/reap)** for REAP (Router-weighted Expert Activation Pruning), the pruning method — [arXiv:2510.13999](https://arxiv.org/abs/2510.13999).
57
+ - **[turboderp / exllamav3](https://github.com/turboderp-org/exllamav3)** for the EXL3 trellis quantization format and runtime.
58
+
59
+ Observations: [`0xSero/glm-5.3-nvfp4-reap-observations-v1`](https://huggingface.co/datasets/0xSero/glm-5.3-nvfp4-reap-observations-v1). Fidelity study: [`0xSero/glm-5.3-reap-fidelity-study`](https://huggingface.co/datasets/0xSero/glm-5.3-reap-fidelity-study). Pruning, quantization, and evaluation ran on 8× NVIDIA RTX PRO 6000 Blackwell.
60
+
61
+ ## Citation
62
+
63
+ ```bibtex
64
+ @misc{lasby2025reap,
65
+ title = {REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression},
66
+ author = {Mike Lasby and Ivan Lazarevich and Nish Sinnadurai and Sean Lie and Yani Ioannou and Vithursan Thangarasa},
67
+ year = {2025}, eprint = {2510.13999}, archivePrefix = {arXiv}
68
+ }
69
+ ```
70
+
71
+ ## License
72
+
73
+ Inherited from the base model — see the [GLM-5.3 model card](https://huggingface.co/zai-org/GLM-5.3) for terms.