---
base_model: zai-org/GLM-5.3-Flash-BF16
library_name: transformers
license: mit
tags:
- glm5
- exl3
- quantized
- mixture-of-experts
---

# GLM-5.3 Flash EXL3 K3

Three-bit EXL3/MCG for every routed expert in
[`zai-org/GLM-5.3-Flash-BF16`](https://huggingface.co/zai-org/GLM-5.3-Flash-BF16),
with everything else retained from the official checkpoint.

## What moved—and what did not

- **K3 EXL3/MCG:** all 288 routed experts' `gate_proj`, `up_proj`, and
  `down_proj` tensors in target layers 3…44 plus checkpoint MTP layer 45.
- **37,152 quantized projections** in one target-plus-MTP calibration run.
- Attention, dense MLPs, shared experts, routers, norms, embeddings, vision,
  and all other tensors remain native precision.
- The release audit compared **1,618 native
  tensors / 18.01 GiB byte-for-byte** against source revision
  `f12e0fe1f6b2ea274c11a569582edfd99d993c5e`.
- Packed checkpoint payload: **127.30 GiB** across
  16 safetensors shards.

Quantization used GPTQModel `0565af7ce20a93df9bbc0e5563d7c6f60916f41a`, EXL3 MCG K3, seed 787,
`sigma_reg=0.025`, automatic output-scale selection, and a fixed 1,426-record
calibration corpus (`sha256:4e569625d97865777da92167b8fbf6fabb4ab7adf55baa5d676c81ce8dd95244`). Natural GLM router
traffic supplied expert Hessians; the committed recovery contract covers any
expert below the 1,024-route floor. The full error ledger and validation report
ship with the model.

Before upload, the fully materialized checkpoint is loaded directly by the
two-GPU B12x/vLLM recipe with adaptive MTP5 and NVFP4 MLA. Publication is gated
on ordinary generation plus five strict tool-call scenarios. The exact prompts,
expected behavior, actual calls, and raw report ship in
`quantization/vllm-prompt-expected-actual.md` and
`quantization/vllm-tool-eval.json`.

## Serving

This is intended for the optimized two-GPU GLM-5.3 recipe at
[`tpurtell/glm-5.3-flash-ext3-4-bit-2x-rtx`](https://github.com/tpurtell/glm-5.3-flash-ext3-4-bit-2x-rtx).
That runtime carries the GLM/EXL3/MTP and B12x ports; this card does not claim
stock-vLLM support.

Huge thanks to **Z.ai** for GLM-5.3 Flash, **Brandon** for the earlier K4 quant
and qualification work that helped inform this run, **MiaAI-Lab** for the nearby
dual-DGX-Spark reference, and the GPTQModel, ExLlamaV3, vLLM, and B12x
contributors.

## Audit identity

- Source index: `sha256:e6007bd58fb7e07f9fe69544257ee2713f252ef5855bbf685b48c991d524ef0f`
- One-shot plan: `sha256:fdce3c2c0078e27b0fa8bdab92cbd13f6c0ab34fb8d3e836fc56b44afc4ec82c`
- Validation: `sha256:625dcdfc8a031506d1406804cc72b7373c600567e076220b756f9504bc0fd284`