Support this work → · X · GitHub · REAP paper · Cerebras REAP

gemma-moe-reap

REAP-pruned the base model.

At a glance


Base model	—
Format	BF16
Total params	—
Active / token	—
Experts / layer	—
Layers	—
Hidden size	—
Context	—
On-disk size	0 GB

Which variant should I pick?

Variant	Format	Link
`Gemma-4-19B`	BF16	link
`Gemma-4-21B`	BF16	link
`gemma-moe-reap` (this)	BF16	link

Model repository for 0xSero/gemma-moe-reap.

License & citation

License inherited from the base model.

@misc{lasby2025reap,
  title  = {REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression},
  author = {Mike Lasby and Ivan Lazarevich and Nish Sinnadurai and Sean Lie and Yani Ioannou and Vithursan Thangarasa},
  year   = {2025}, eprint = {2510.13999}, archivePrefix = {arXiv}
}