Instructions to use wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1")
model = AutoModelForMultimodalLM.from_pretrained("wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1

SGLang

How to use wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1 with Docker Model Runner:
```
docker model run hf.co/wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3.25-v1
```

GLM-5.3-Flash-EXL3-K3.25-v1 / quantization /validation-report.json

wrldsuksgo2mars

Add files using upload-large-folder tool

a4480b6 verified about 1 month ago

Raw History Blame Contribute Delete

2.98 kB

	{
	"ledger": {
	"complete_family_records": 12384,
	"k4_by_projection": {
	"down_proj": 4536,
	"gate_proj": 1701,
	"up_proj": 2835
	},
	"ledger": "ds4rt-exl3-error-ledger.jsonl",
	"ledger_sha256": "7788785b0af5af259ecff32c64ef2d962452d8bfff90f764603db8846279dd03",
	"projection_records": 37152
	},
	"native": {
	"aggregate_sha256": "5dc4c673d57bd1254d75ff22fb0dff9272c5325f686b99e6de24cd08f5dfbe22",
	"byte_identical": true,
	"bytes": 19339524984,
	"tensor_count": 1618
	},
	"output": {
	"bytes": 146198940952,
	"index_sha256": "1ddfc0b2447bbe5d5b3ce3648baa62a01b49247579581e2643674e957aa1e103",
	"shards": 18,
	"tensor_count": 150226
	},
	"quantization_config": {
	"alias": "quantization_config.json",
	"bytes": 32772423,
	"codebook": "mcg",
	"compact_model_config": {
	"architectures": [
	"Glm5NextForConditionalGeneration"
	],
	"bytes": 13180,
	"embedded_manifest": false,
	"maximum_bytes": 262144,
	"model_type": "glm5_next",
	"path": "config.json",
	"sha256": "6b477cfc1fbf8cdf3795c6389bc9712503e3f7c3889145036488ffab2b1a7781"
	},
	"k4_by_projection": {
	"down_proj": 4536,
	"gate_proj": 1701,
	"up_proj": 2835
	},
	"modules": 37152,
	"mtp_bits": 3,
	"path": "quantize_config.json",
	"sha256": "fd116534bff1c7cb7c319be03e1f4d14d9faafa0d36ef95d37eebe40ee29c429",
	"target_bits": {
	"average": 3.25,
	"base": 3,
	"upgrade": 4
	}
	},
	"quantized": {
	"codebook": "mcg",
	"experts_per_layer": 288,
	"k4_by_projection": {
	"down_proj": 4536,
	"gate_proj": 1701,
	"up_proj": 2835
	},
	"layers": [
	3,
	45
	],
	"modules": 37152,
	"mtp_layer": 45,
	"mtp_trellis_k": 3,
	"projections_per_expert": [
	"gate_proj",
	"up_proj",
	"down_proj"
	],
	"stored_tensors": 148608,
	"target_average_bpw": 3.25,
	"target_trellis_k": [
	3,
	4
	]
	},
	"run": {
	"complete_sha256": "e637bae4c59e16a3cec8bc20d03d990db1459a4f02b1c682a2d0d88ec838753e",
	"k4_by_projection": {
	"down_proj": 4536,
	"gate_proj": 1701,
	"up_proj": 2835
	},
	"mtp_bits": 3,
	"output": "/mnt/scratch/GLM-5.3-Flash-EXL3-K3.25-v1",
	"plan_sha256": "cb8ee0b49c6a41bb31e9813214193fa9426c209dd0a0ca9f52ce8b5ab61465d8",
	"projection_checkpoint_manifests": 46224,
	"projection_checkpoint_payloads": 46224,
	"tier_plan_sha256s_sha256": "44f89b9daf654c99f0c29b46d2002de109b4d49f1fbecaa569dbd30e938379ce",
	"tier_plans": 42
	},
	"schema": "glm53-flash-exl3-k325-validation-v1",
	"source": {
	"bytes": 642652070880,
	"index_sha256": "e6007bd58fb7e07f9fe69544257ee2713f252ef5855bbf685b48c991d524ef0f",
	"revision": "f12e0fe1f6b2ea274c11a569582edfd99d993c5e",
	"shards": 120,
	"tensor_count": 38770
	},
	"validation_sha256": "30e5f8f5fc6c1d10cdd17749b40559a0dca89bec89b5affad219cab0f0260160"
	}