Standardize model card (template rollout)

README.md

@@ -3,28 +3,52 @@ language:
 - en
 license: gemma
 tags:
-- safetensors
+- cerebras
+- expert-pruning
+- gemma
 - gemma4
 - moe
 - pruning
 - reap
-- cerebras
-- expert-pruning
+- safetensors
 base_model:
 - google/gemma-4-26b-a4b-it
 library_name: transformers
 pipeline_tag: text-generation
 ---
+
 > [!TIP]
-> Support this work: **[donate.sybilsolutions.ai](https://donate.sybilsolutions.ai)**
-> 
-> REAP surfaces: [GLM](https://huggingface.co/spaces/0xSero/reap-glm-family) | [MiniMax](https://huggingface.co/spaces/0xSero/reap-minimax-family) | [Qwen](https://huggingface.co/spaces/0xSero/reap-qwen-family) | [Gemma](https://huggingface.co/spaces/0xSero/reap-gemma-family) | [Paper](https://arxiv.org/abs/2510.13999) | [Code](https://github.com/CerebrasResearch/reap) | [PR17](https://github.com/CerebrasResearch/reap/pull/17) | [Cerebras Collection](https://huggingface.co/collections/cerebras/cerebras-reap)
+> **[Support this work →](https://donate.sybilsolutions.ai)** · [X](https://x.com/0xsero) · [GitHub](https://github.com/0xsero) · [REAP paper](https://arxiv.org/abs/2510.13999) · [Cerebras REAP](https://huggingface.co/collections/cerebras/cerebras-reap)
+
+# Gemma-4-21B
+
+REAP-pruned [google/gemma-4-26b-a4b-it](https://huggingface.co/google/gemma-4-26b-a4b-it).
+
+## At a glance
+
+| | |
+|---|---|
+| Base model | — |
+| Format | BF16 |
+| Total params | **21B** |
+| Active / token | — |
+| Experts / layer | 103 |
+| Layers | — |
+| Hidden size | — |
+| Context | — |
+| On-disk size | 43 GB |
 
-# Gemma 4 21B-A4B-it REAP
+## Which variant should I pick?
+
+| Variant | Format | Link |
+|---|---|---|
+| `Gemma-4-19B` | BF16 | [link](https://huggingface.co/0xSero/Gemma-4-19B) |
+| `Gemma-4-21B` **(this)** | BF16 | [link](https://huggingface.co/0xSero/Gemma-4-21B) |
+| `gemma-moe-reap` | BF16 | [link](https://huggingface.co/0xSero/gemma-moe-reap) |
 
 **20% expert-pruned** version of [google/gemma-4-26b-a4b-it](https://huggingface.co/google/gemma-4-26b-a4b-it) using **[Cerebras REAP](https://github.com/cerebras/reap)** (Router-weighted Expert Activation Pruning).
 
-| | Original | This Model (0.20) | [0.30 variant](https://huggingface.co/0xSero/gemma-4-19b-a4b-it-REAP) |
+| | Original | This Model (0.20) | [0.30 variant](https://huggingface.co/0xSero/Gemma-4-19B) |
 |---|---:|---:|---:|
 | **Total params** | ~26B | **21.34B** | 19.02B |
 | **Experts per layer** | 128 | **103** | 90 |
@@ -131,7 +155,7 @@ Gemma 4 uses a hybrid sliding/full attention MoE architecture:
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
-model_id = "0xSero/gemma-4-21b-a4b-it-REAP"
+model_id = "0xSero/Gemma-4-21B"
 tokenizer = AutoTokenizer.from_pretrained(model_id)
 model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto", trust_remote_code=True)
 
@@ -147,7 +171,7 @@ print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_toke
 ```bash
 pip install vllm>=0.19 transformers>=5.0
 
-vllm serve 0xSero/gemma-4-21b-a4b-it-REAP \
+vllm serve 0xSero/Gemma-4-21B \
     --tensor-parallel-size 2 \
     --enforce-eager \
     --gpu-memory-utilization 0.9 \
@@ -155,31 +179,24 @@ vllm serve 0xSero/gemma-4-21b-a4b-it-REAP \
     --trust-remote-code
 ```
 
-## Citation
-
-```bibtex
-@inproceedings{lasby2025reap,
-  title={{REAP} the Experts: Why Pruning Prevails for One-Shot {MoE} Compression},
-  author={Lasby, Mike and others},
-  booktitle={International Conference on Learning Representations (ICLR)},
-  year={2026},
-  url={https://arxiv.org/abs/2510.13999}
-}
-```
 
 ## Links
 
 - **REAP paper:** [arxiv.org/abs/2510.13999](https://arxiv.org/abs/2510.13999)
 - **REAP code:** [github.com/cerebras/reap](https://github.com/cerebras/reap)
-- **30% pruned variant:** [0xSero/gemma-4-19b-a4b-it-REAP](https://huggingface.co/0xSero/gemma-4-19b-a4b-it-REAP)
+- **30% pruned variant:** [0xSero/Gemma-4-19B](https://huggingface.co/0xSero/Gemma-4-19B)
 - **Base model:** [google/gemma-4-26b-a4b-it](https://huggingface.co/google/gemma-4-26b-a4b-it)
 
-## Sponsors
+## License & citation
+License inherited from the base model.
 
-Thank you for the kind sponsors, wouldn't be possible without them:
+```bibtex
+@misc{lasby2025reap,
+  title  = {REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression},
+  author = {Mike Lasby and Ivan Lazarevich and Nish Sinnadurai and Sean Lie and Yani Ioannou and Vithursan Thangarasa},
+  year   = {2025}, eprint = {2510.13999}, archivePrefix = {arXiv}
+}
+```
 
-- Nvidia
-- TNG Technology
-- Lambda
-- Prime Intellect
-- HotAisle
+## Sponsors
+Made possible by **NVIDIA · TNG Technology · Lambda · Prime Intellect · Hot Aisle**.