---
license: mit
base_model: zai-org/GLM-5.2-FP8
base_model_relation: quantized
language:
  - en
  - zh
tags:
  - glm_moe_dsa
  - int4
  - cpu
  - moe
  - expert-streaming
  - colibri
---

# GLM-5.2 — colibrì int4 container (~370 GB)

Pre-converted weights for **[colibrì](https://github.com/JustVugg/colibri)** — the pure-C engine that runs GLM-5.2 (744B MoE) on a consumer machine with ~25 GB of RAM by streaming routed experts from disk.

This is the output of colibrì's `coli convert` (`convert_fp8_to_int4.py --ebits 4 --io-bits 8`, including the MTP head for native speculative decoding), uploaded so you don't have to download the 756 GB FP8 checkpoint and spend a day converting it.

**⚠️ This is NOT a GGUF / AWQ / GPTQ / MLX model.** It is colibrì's own container: for each quantized weight, `name` (U8, packed int4 nibbles) + `name.qs` (F32 per-row scales), quantized with math bit-identical to the engine's C kernels. It only works with the colibrì engine.

## Usage

```bash
# get the engine
git clone https://github.com/JustVugg/colibri && cd colibri/c && ./setup.sh

# download this repo to a FAST local disk (NVMe, ext4 — never a network/9p mount)
hf download jlnsrk/GLM-5.2-colibri-int4 --local-dir /nvme/glm52_i4

# chat (RAM budget, expert cache and MTP auto-detected)
COLI_MODEL=/nvme/glm52_i4 ./coli chat
```

Requirements: Linux (or WSL2), gcc + OpenMP, AVX2, ≥16 GB RAM, ~400 GB free NVMe.

## What's inside

| file | contents |
|---|---|
| `out-*.safetensors` | dense weights (attention/MLA, shared experts, embeddings) + 21,504 routed experts, int4 per-row scales; router/norms kept F32 |
| MTP shard | GLM-5.2's multi-token-prediction head (layer 78) — enables lossless speculative decoding (~2 tok/forward) |
| `config.json`, `tokenizer*.json`, `generation_config.json` | copied from the base repo |

Conversion: FP8 (e4m3, 128×128 block scales) → f32 → int4 with `np.rint` matching the engine's `lrintf` — token-identical to converting locally.

## Provenance & license

Converted from [zai-org/GLM-5.2-FP8](https://huggingface.co/zai-org/GLM-5.2-FP8) (MIT). This derivative is likewise MIT. Conversion performed with colibrì's official converter, unmodified.