---
license: mit
base_model: zai-org/GLM-5.3-Flash-BF16
base_model_relation: quantized
pipeline_tag: image-text-to-text
tags:
  - exl3
  - mixture-of-experts
  - dgx-spark
  - multimodal
---

# GLM-5.3-Flash EXL3 TR3 2.0bpw

This is a selectively compressed GLM-5.3-Flash checkpoint validated on one
NVIDIA DGX Spark. It serves text, structured tool calls, images, native video,
and a 200k-token request from one GB10 GPU with CUDA graphs enabled.

The routed experts use the EXL3 K2 tier. The `2.0bpw` name describes that
routed-expert tier; it is not the storage rate of every tensor. Routing,
attention, embeddings, vision, and other retained components are preserved in
their recorded formats.

## Artifact

| Property | Value |
|---|---:|
| Base model | `zai-org/GLM-5.3-Flash-BF16` |
| Base revision | `a6c167b62691b2bac901344b65cb651a70f53e43` |
| Weight files | 133 root-level safetensors shards |
| Weight bytes | 111,352,026,456 |
| Indexed tensors | 583,090 |
| Routed-expert tier | EXL3 K2, 2.0 bpw |
| Structural manifest SHA-256 | `501641b947fa56afc9ed098bdc034e59c2bd229d4fa1a1d7b80e3727f10c8ba3` |

The checkpoint includes an unused MTP companion layer (889 tensors,
14,865,185,408 bytes). The accepted baseline does not load that layer. It is
retained for provenance and future runtime work; enabling it requires a new
memory and acceptance run.

## Measured one-Spark baseline

Tested on one DGX Spark / one GB10 GPU with the custom vLLM EXL3 loader, TP1,
FP8 target KV cache, 204,800 configured context, prefix caching disabled, and
full-decode CUDA graphs.

| Gate | Result |
|---|---|
| Live model load | 89.89 GiB |
| Live KV capacity | 1,638,400 tokens on the final running instance (8.00x configured context) |
| Text identity | Exact final content and natural stop passed |
| Structured tools | Parsed function name and JSON arguments passed |
| Images | 4/4 paired synthetic fixtures passed exact JSON checks |
| Native video | 2/2 paired synthetic fixtures passed exact JSON checks |
| 200k context | 200,012 server-reported prompt tokens; exact 4/4 retrieval at 5%, 35%, 65%, and 95%; natural stop |
| Multilingual smoke | Arabic, Chinese, and Polish constraint-following prompts passed |

The 200k retrieval took 498.40 seconds end to end, including 474.04 seconds to
the first streamed token. Its client-side decode estimate was 9.28 tokens/s.
No prefix-cache hit was used.

Bounded exact-answer speed cells measured:

| Input text tokens | Incoming concurrency | Wall time | Client decode estimate |
|---:|---:|---:|---:|
| 1,024 | 1 | 14.88 s | 9.16 tok/s |
| 32,768 | 1 | 84.45 s | 9.47 tok/s |
| 131,072 | 1 | 306.10 s | 9.28 tok/s |
| 1,024 | 2 | 29.84 s total | 9.34 tok/s mean |

These are client estimates from usage counts and stream timing, not isolated
kernel timings. The server was intentionally configured for one active
sequence, so two incoming requests queued rather than decoding concurrently.

## Important limitations

- The requested 25–50 tok/s decode target was **not met**. The accepted
  baseline is consistently about 9.3 tok/s.
- One simple Python task spent the full 600-second wall deadline generating
  reasoning and returned no final content. This is retained as a failed
  responsiveness/code-quality probe.
- Exact held-out KLD against the BF16 base has not been measured for this exact
  2.0-bpw artifact. Historical results from other prunes are not claimed here.
- The visual checks are small controlled fixtures, not a broad vision-quality
  benchmark.
- DFlash2 acceleration was investigated but is not part of this release. In
  the tested vLLM build, draft metadata inherited the target MLA KV format and
  failed initialization. Context was not reduced and eager mode was not used
  as a workaround.

## Runtime

This checkpoint needs the accompanying rank-stacked EXL3 runtime. Stock vLLM
does not understand this artifact layout. Use the pinned ARM64 container and
launch recipe from the public reproduction repository linked in this model's
repository metadata. Mount weights read-only. CUDA graphs are required.

## Method and attribution

- Z.AI: GLM-5.3-Flash base model and tokenizer.
- TurboDerp / ExLlamaV3: EXL3 format and encoder lineage.
- Brandon M. Music: MIT-licensed GLM-5.2 EXL3/TR3 predecessor and
  reproducibility methods. The permitted reference revision is
  `f79c9167690ca705e877ae4dc55a841d1aae1247`; later GLM-5.3 artifacts were not
  used.
- MiaAI Lab: GLM-5.3-Flash DGX Spark runtime lineage.
- Cerebras Research: REAP observation and expert-selection lineage.

Private prompts, credentials, hostnames, addresses, and fleet logs are not
included in this repository.