---
language:
- en
- zh
library_name: mlx
pipeline_tag: text-generation
tags:
- mlx
license: other
license_name: glm-5.3
license_link: https://huggingface.co/zai-org/GLM-5.3-BF16/blob/main/LICENSE
base_model: zai-org/GLM-5.3-BF16
base_model_relation: quantized
---

# kernelpool/GLM-5.3-5bit-UVMAX

Mixed-precision (UVMAX) quantization of
[zai-org/GLM-5.3-BF16](https://huggingface.co/zai-org/GLM-5.3-BF16),
converted with mlx-lm from the bf16 release.

## What is UVMAX?

UVMAX assigns bit widths per tensor class instead of quantizing uniformly.

| tensor class | precision | parameters | size | share |
|---|---|---|---|---|
| Expert FFN gate/up | 4-bit, group 128 | 483B | 239.1 GiB | 59.1% |
| Expert FFN down | 5-bit, group 128 | 242B | 147.7 GiB | 36.5% |
| Attention + DSA indexer + dense MLP | 8-bit, group 64 | 13.7B | 13.6 GiB | 3.4% |
| Shared experts | 8-bit, group 64 | 2.8B | 2.8 GiB | 0.7% |
| Embeddings, LM head | 4-bit, group 64 | 1.9B | 1.0 GiB | 0.2% |
| Routers | bf16 | 0.1B | 0.2 GiB | <0.1% |
| Norms | bf16 | — | — | <0.1% |
| total | 4.67 bits/weight | 743B | 404 GiB | |

## Quality

Teacher-forced against the bf16 release on identical tokens, 48 windows of
1025 tokens. KLD is `KL(bf16 ‖ quant)` over the full output distribution and
is corpus-specific.

**UVMAX (4.67 bits/weight)**

| corpus | ppl bf16 | ppl UVMAX | ratio | mean KLD | median KLD | top-1 agreement |
|---|---|---|---|---|---|---|
| Linux kernel C | 1.434 | 1.459 | 1.02× | 0.040 | 0.0002 | 96.8% |
| XNU kernel C | 2.435 | 2.472 | 1.02× | 0.056 | 0.0027 | 93.4% |
| JavaScriptCore C++ | 1.757 | 1.791 | 1.02× | 0.055 | 0.0007 | 95.1% |
| English prose | 2.687 | 2.750 | 1.02× | 0.063 | 0.0076 | 92.4% |
| all | 2.015 | 2.053 | 1.02× | 0.053 | 0.0012 | 94.4% |

**Uniform 4-bit (4.50 bits/weight)**

| corpus | ppl bf16 | ppl 4-bit | ratio | mean KLD | median KLD | top-1 agreement |
|---|---|---|---|---|---|---|
| Linux kernel C | 1.434 | 1.492 | 1.04× | 0.076 | 0.0004 | 95.3% |
| XNU kernel C | 2.435 | 2.522 | 1.04× | 0.109 | 0.0063 | 91.0% |
| JavaScriptCore C++ | 1.757 | 1.841 | 1.05× | 0.102 | 0.0014 | 93.4% |
| English prose | 2.687 | 2.909 | 1.08× | 0.143 | 0.0209 | 88.6% |
| all | 2.015 | 2.119 | 1.05× | 0.107 | 0.0028 | 92.1% |

## Use with mlx

Requires a recent mlx-lm with GLM-5.3 (`glm_moe_dsa`) support.

The model fits on a single 512 GB machine:

```bash
mlx_lm.server --model kernelpool/GLM-5.3-5bit-UVMAX
```

Sampling follows the base model: temperature 1.0, top-p 0.95.