Instructions to use wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1")
model = AutoModelForMultimodalLM.from_pretrained("wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1

SGLang

How to use wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1 with Docker Model Runner:
```
docker model run hf.co/wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1
```

GLM-5.3 Flash EXL3 K3.25

Projection-aware K3/K4 EXL3/MCG for the routed experts in zai-org/GLM-5.3-Flash-BF16, with everything else retained from the official checkpoint.

What moved—and what did not

K3.25 target experts: layers 3…44 start at K3, then exactly 9,072 of 36,288 projections move to K4. The promotion budget is gate/up/down = 1,701 / 2,835 / 4,536 (the 3:5:8 quality allocation).
Straight-K3 MTP: all 864 checkpoint-MTP projections in layer 45 stay K3. It remains valid for diagnostics, but the serving recipe defaults to DFlash2, so extra K4 bytes and search time are not spent there.
37,152 final quantized projections were produced in one target-plus-MTP model.quantize(...) call.
Attention, dense MLPs, shared experts, routers, norms, embeddings, vision, and all other tensors remain native precision.
The release audit compared 1,618 native tensors / 18.01 GiB byte-for-byte against source revision f12e0fe1f6b2ea274c11a569582edfd99d993c5e.
Packed checkpoint payload: 136.16 GiB across 18 safetensors shards.

Quantization used GPTQModel a053382584fa58cba7bf212ef1b829d08b29b2c0, EXL3 MCG adjacent tiers, seed 787, sigma_reg=0.025, automatic output-scale selection, and a fixed 1,426-record calibration corpus (sha256:4e569625d97865777da92167b8fbf6fabb4ab7adf55baa5d676c81ce8dd95244). Within every target layer, K4 promotion follows measured Hessian-weighted relative error times natural gate-squared mass while the global 3:5:8 budget stays exact. Natural GLM router traffic supplied expert Hessians; the committed recovery contract covers any expert below the 1,024-route floor. Allocation provenance and the validation report ship with the model; the full error ledger is retained separately as internal quantization evidence.

Qualified two-GPU serving result

Release v0.7.0 of the matching B12x/vLLM recipe qualified the public target revision with DFlash2 K5, FP8 MLA, TP2 + EP2 + DCP2, vision up to 16 images, and a 1,048,576-token request limit on 2× RTX PRO 6000 Blackwell at a 400 W power cap per GPU.

Code-agent decode: 213 tok/s at C1 and 832 tok/s at C16.
128K cold prefill: 4,572 tok/s.
Available KV cache: 2,758,919 tokens.
Exact 1M multi-needle test: 6/6 needles recovered.
Exact 69-case thinking tool-call suite at parallelism 8: 86/100 (55 pass, 8 partial, 6 fail), versus 88/100 for the matched uniform-K3 control.

The full prompts, outputs, scoring, performance curves, and machine-readable receipts are in the recipe's v0.7.0 benchmark report.

Serving

This is intended for the optimized two-GPU GLM-5.3 recipe at tpurtell/glm-5.3-flash-ext3-4-bit-2x-rtx. That runtime carries the GLM/EXL3/MTP and B12x ports; this card does not claim stock-vLLM support.

Huge thanks to Z.ai for GLM-5.3 Flash, Brandon for the earlier K4 quant and qualification work that helped inform this run, MiaAI-Lab for the nearby dual-DGX-Spark reference, and the GPTQModel, ExLlamaV3, vLLM, and B12x contributors.

Audit identity

Source index: sha256:e6007bd58fb7e07f9fe69544257ee2713f252ef5855bbf685b48c991d524ef0f
One-shot plan: sha256:cb8ee0b49c6a41bb31e9813214193fa9426c209dd0a0ca9f52ce8b5ab61465d8
Validation: sha256:30e5f8f5fc6c1d10cdd17749b40559a0dca89bec89b5affad219cab0f0260160

Downloads last month: 305

Safetensors

Model size

73B params

Tensor type

BF16

F32

F16

I16

Model tree for wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1

Base model

zai-org/GLM-5.3-Flash-BF16

Quantized

(33)

this model

Finetunes

1 model