---
language:
- en
library_name: transformers
tags:
- 4bit
- MOE
- autoround
- cerebras
- glm
- glm4
- pruning
- quantized
- reap
- w4a16
license: mit
pipeline_tag: text-generation
base_model:
- cerebras/GLM-4.6-REAP-218B-A32B
quantized_by: 0xSero
base_model_relation: quantized
---

> [!TIP]
> **[Support this work →](https://donate.sybilsolutions.ai)** · [X](https://x.com/0xsero) · [GitHub](https://github.com/0xsero) · [REAP paper](https://arxiv.org/abs/2510.13999) · [Cerebras REAP](https://huggingface.co/collections/cerebras/cerebras-reap)

# GLM-4.6-218B-W4A16

W4A16 quantization of [cerebras/GLM-4.6-REAP-218B-A32B](https://huggingface.co/cerebras/GLM-4.6-REAP-218B-A32B).

## At a glance

| | |
|---|---|
| Base model | [cerebras/GLM-4.6-REAP-218B-A32B](https://huggingface.co/cerebras/GLM-4.6-REAP-218B-A32B) |
| Format | W4A16 |
| Total params | **218B** |
| Active / token | 32B |
| Experts / layer | 96 |
| Layers | 92 |
| Hidden size | 5120 |
| Context | 202,752 |
| On-disk size | 119 GB |

<p align="center">
  <em>𓌳 <strong>REAP</strong>𓌳  the Experts reduce  </em><br>
  <a href="https://arxiv.org/abs/2510.13999">📄 Paper</a> • <a href="https://github.com/CerebrasResearch/reap">💻 Code</a>
</p>

# GLM-4.6-REAP-218B-A32B-W4A16-AutoRound

**W4A16 quantized** version of Cerebras' official [GLM-4.6-REAP-218B-A32B](https://huggingface.co/cerebras/GLM-4.6-REAP-218B-A32B).

- **~4x Size Reduction**: ~436GB → ~116GB
- **Runs on Consumer Hardware**: 8x RTX 3090 or 4x RTX 4090
- **vLLM/SGLang Compatible**: Drop-in deployment


## 📋 Model Specifications

| Property | Value |
|----------|-------|
| **Base Model** | [cerebras/GLM-4.6-REAP-218B-A32B](https://huggingface.co/cerebras/GLM-4.6-REAP-218B-A32B) |
| **Parameters** | 218B total, 32B activated |
| **Quantization** | W4A16 (4-bit weights, 16-bit activations) |
| **Original Size** | ~436GB |
| **Quantized Size** | ~116GB |

---

## 🔬 Calibration Dataset: Deep Dive

REAP's effectiveness depends critically on **calibration data that represents the target use case**. We specifically optimized for **code generation**, **function/tool calling**, and **agentic workflows**.

### Why These 3 Datasets?

| Dataset | Samples | Purpose | Why It Matters |
|---------|---------|---------|----------------|
| [evol-codealpaca-v1](https://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1) | 700 | Code generation | **51% of mix** — Code tasks activate specific expert pathways; pruning without code calibration destroys coding ability |
| [xlam-function-calling-60k](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k) | 330 | Function/tool calling | **24% of mix** — Tool use requires structured JSON output; experts handling schema generation must be preserved |
| [SWE-smith-trajectories](https://huggingface.co/datasets/SWE-bench/SWE-smith-trajectories) | 330 | Agentic multi-turn | **24% of mix** — Real SWE-bench trajectories with tool calls, file edits, and multi-step reasoning |

### The Science Behind Dataset Selection

```
REAP Algorithm:
1. Forward pass calibration samples through model
2. Record which experts activate and their magnitudes
3. Compute saliency = router_weight × activation_norm
4. Prune lowest-saliency experts

Key Insight: Experts are TASK-SPECIFIC
├── Some experts specialize in natural language
├── Some experts specialize in code syntax
├── Some experts specialize in JSON/structured output
└── Some experts specialize in multi-turn context

If calibration lacks code → code-specialized experts appear "unused" → get pruned → model loses coding ability
```

### Cerebras' Original Mix (from paper)

Cerebras used the same 3 datasets in their GLM-4.6 REAP experiments:
- evol-codealpaca-v1 for code generation
- xlam-function-calling-60k for tool calling
- SWE-smith-trajectories for agentic tasks

We followed this exact recipe for reproducibility.

### Combined Dataset

Our calibration mix: [0xSero/glm47-reap-calibration-v2](https://huggingface.co/datasets/0xSero/glm47-reap-calibration-v2)


---

## License & citation
License inherited from the base model.

```bibtex
@misc{lasby2025reap,
  title  = {REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression},
  author = {Mike Lasby and Ivan Lazarevich and Nish Sinnadurai and Sean Lie and Yani Ioannou and Vithursan Thangarasa},
  year   = {2025}, eprint = {2510.13999}, archivePrefix = {arXiv}
}
```

## Sponsors
Made possible by **NVIDIA · TNG Technology · Lambda · Prime Intellect · Hot Aisle**.