Instructions to use brandonmusic/GLM-5.3-Flash-tr3-4bpw with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use brandonmusic/GLM-5.3-Flash-tr3-4bpw with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="brandonmusic/GLM-5.3-Flash-tr3-4bpw")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("brandonmusic/GLM-5.3-Flash-tr3-4bpw")
model = AutoModelForMultimodalLM.from_pretrained("brandonmusic/GLM-5.3-Flash-tr3-4bpw", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use brandonmusic/GLM-5.3-Flash-tr3-4bpw with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "brandonmusic/GLM-5.3-Flash-tr3-4bpw"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brandonmusic/GLM-5.3-Flash-tr3-4bpw",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/brandonmusic/GLM-5.3-Flash-tr3-4bpw

SGLang

How to use brandonmusic/GLM-5.3-Flash-tr3-4bpw with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "brandonmusic/GLM-5.3-Flash-tr3-4bpw" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brandonmusic/GLM-5.3-Flash-tr3-4bpw",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "brandonmusic/GLM-5.3-Flash-tr3-4bpw" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brandonmusic/GLM-5.3-Flash-tr3-4bpw",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use brandonmusic/GLM-5.3-Flash-tr3-4bpw with Docker Model Runner:
```
docker model run hf.co/brandonmusic/GLM-5.3-Flash-tr3-4bpw
```

brandonmusic commited on Aug 27

Commit

df82110

verified ·

1 Parent(s): de256db

Add files using upload-large-folder tool

Browse files

This view is limited to 50 files because it contains too many changes. See raw diff

Files changed (50) hide show

.gitattributes +1 -0
model-00007-of-00120.safetensors +3 -0
model-00038-of-00120.safetensors +3 -0
model-00039-of-00120.safetensors +3 -0
model-00040-of-00120.safetensors +3 -0
model-00041-of-00120.safetensors +3 -0
model-00042-of-00120.safetensors +3 -0
model-00043-of-00120.safetensors +3 -0
model-00044-of-00120.safetensors +3 -0
model-00045-of-00120.safetensors +3 -0
model-00046-of-00120.safetensors +3 -0
model-00047-of-00120.safetensors +3 -0
model-00048-of-00120.safetensors +3 -0
model-00049-of-00120.safetensors +3 -0
model-00050-of-00120.safetensors +3 -0
model-00051-of-00120.safetensors +3 -0
model-00052-of-00120.safetensors +3 -0
model-00053-of-00120.safetensors +3 -0
model-00054-of-00120.safetensors +3 -0
model-00055-of-00120.safetensors +3 -0
model-00056-of-00120.safetensors +3 -0
model-00057-of-00120.safetensors +3 -0
model-00058-of-00120.safetensors +3 -0
model-00059-of-00120.safetensors +3 -0
model-00060-of-00120.safetensors +3 -0
model-00061-of-00120.safetensors +3 -0
model-00062-of-00120.safetensors +3 -0
model-00063-of-00120.safetensors +3 -0
model-00064-of-00120.safetensors +3 -0
model-00065-of-00120.safetensors +3 -0
model-00066-of-00120.safetensors +3 -0
model-00067-of-00120.safetensors +3 -0
model-00068-of-00120.safetensors +3 -0
model-00069-of-00120.safetensors +3 -0
model-00070-of-00120.safetensors +3 -0
model-00071-of-00120.safetensors +3 -0
model-00072-of-00120.safetensors +3 -0
model-00073-of-00120.safetensors +3 -0
model-00074-of-00120.safetensors +3 -0
model-00075-of-00120.safetensors +3 -0
model-00076-of-00120.safetensors +3 -0
model-00077-of-00120.safetensors +3 -0
model-00078-of-00120.safetensors +3 -0
model-00079-of-00120.safetensors +3 -0
model-00080-of-00120.safetensors +3 -0
model-00081-of-00120.safetensors +3 -0
model-00082-of-00120.safetensors +3 -0
model-00083-of-00120.safetensors +3 -0
model-00084-of-00120.safetensors +3 -0
model-00085-of-00120.safetensors +3 -0

.gitattributes CHANGED Viewed

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text
 model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text

 *tfevents* filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text
 model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
+quantization_config.json filter=lfs diff=lfs merge=lfs -text

model-00007-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a8e2cf2d869afb825934ad43bcbbf7e16132bdfca8443ca52a39c46b36f775cd
+size 1570455356

model-00038-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:26b70b8b61940befb31f4bf5956dfd9dc89500bddb36b4e79eb046ab39000412
+size 1346273240

model-00039-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c2599dd1cdc04c23c3383ecccad7d50d11d0e52c6602574bc1da2428e84f7028
+size 1346272864

model-00040-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e109dbf65aba7827809cc67334065364a1684715179b31311e0a31c00d8dd48c
+size 1570455044

model-00041-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bb85b908911687c0e276e43e81b5a4a265a87c00965324532681097ca491bc9f
+size 1346273248

model-00042-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a017885e30064729e5f70cbaeda434663c6c382aa9d2ba2a73f820a8cd5b6bbe
+size 1346272544

model-00043-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:04f3aeccb23c779ac9e6be9e952418b95cac19327e7873dce5f69e4ad728f6e0
+size 1591901088

model-00044-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fa5a385f7a8220ed99bc081f2fe26a07c480a3da7f497be57cf95f96ceb1c34c
+size 1346273256

model-00045-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b0b508dbd6217fe861a68bd8890415fc0f97404ed9ea7155adae9476b12c27d0
+size 1591900384

model-00046-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c0e27a8d0a9e4521e96f17ad0e40de8152ddd8c26f7739520cc052069e4432f7
+size 1346273248

model-00047-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5c471e0726598af910a1d5c895eca9bf41afe1e45311ee0dc962572b21097f2d
+size 1346273192

model-00048-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1d957114322b3d9dbb4cca73d9bba9cf628ddc7f5a6dd1d6af6224eb6aff003e
+size 1591900416

model-00049-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:895d22c61a559667d84ac65dcd3012de18103738d3660c0c84db0b90e29c7e32
+size 1346273256

model-00050-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c2bd099b71e35b8af36785f23e025c8b3ce393197b801cd3e53ee33e9be71531
+size 1346272920

model-00051-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1c30583b2edd80ed55db4086e1b846f7a9de54f7996bfa07e2ac0039f702c10a
+size 1570454972

model-00052-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7c800f2f476cff08606cc3b036571d43ea501518fcba658aeda5db0fce815346
+size 1346273248

model-00053-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:43acf4bbb16dda6f5a7c31240108bad19c64ac5eb9d20022321c4eeea48bc991
+size 1346272616

model-00054-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:98bb70809af0d4d3177a377eefcbf728dd7fbd9e3891f130c05c66d29cd04860
+size 1591901024

model-00055-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:285c605b3391fbca1a40f28fdb6c1856a155ccfe6de43eb59e11033360dadf51
+size 1346273240

model-00056-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6f825c25cd071930e7cbec832fe71a65767024b6853bdf554a9dd6534d0cd25c
+size 1439345368

model-00057-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9c7e6d5fd6d4138fde51eef51a83ff8a33424294c44d52d1b62b42ddb79ada4d
+size 1494623036

model-00058-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0b73d4dc1f373046f08a000d60484a7cf5335c4b0a1e4588d0cf8089f3ed75f2
+size 1346271976

model-00059-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c5e83b54c3f1912aa5fd35f1c3170415f54708887f6f358cb1eac0b93391394d
+size 1570453660

model-00060-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c37b7e82d16a82243664ecac9e417a815195e1c26d9affe2ce681bffcb2fa046
+size 1346273240

model-00061-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ec26e65bf26f2dadf4e4c7aeefe48acbd9b677a3d03ba0d3229f1cc796647bb4
+size 1346272976

model-00062-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:98c6427e9b22529ea985aa5d343e24055d3fd570b866e4d8dfb8f0b9470caedb
+size 1591900664

model-00063-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dacdd78b8a4f74ea5502d28b0a8d0c632a05252e095abc6245f0a63bdbd57a46
+size 1346273248

model-00064-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7f134d623d5f229cb4732905ceb269404e1b00a72b8c25cc4f09d1e8610aabc5
+size 1346272664

model-00065-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:16661291a3d218f765edcb3831ff9426af21d6d56e6d8082b194e5cac9348109
+size 1570455228

model-00066-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e8c5392e733bd98d5d9f38d47e18cf6834bcbc0b8d24937c42d36ebc60604910
+size 1346273256

model-00067-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f2fb7876facab2bbb569fe94ab41dd1e71c50a3811853bf37688e5ebdc5813c0
+size 1346272368

model-00068-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:abc3009f10522e97eedbcd34903e3aafa55f7d72efa4ff3db4b5c4fb28259ca1
+size 1591901272

model-00069-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ee4f3cb631ca5b0552fab89a3ef05ed0d8dab5ee0066fa245af479c962071e53
+size 1346273240

model-00070-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8b60971fe302e1bcf1bb7da900276a3f26fcd38b10d6760330ee59ea8df7a589
+size 1591900360

model-00071-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9fb9f3581b0a09d43e94e0476b9fa0971b3ab30b3d980bf6bb0814fb92db82d4
+size 1346273256

model-00072-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:caec81374ed646f0d7953f81565438ce18dfc11197cbb1a442b557cfb907dd23
+size 1346273040

model-00073-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:313e11a85717e5fe56c6901f16c10aab760619b7655d355185a17dcb6986397a
+size 1591900600

model-00074-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3c22a6012bb34c264869fd8a93ac019175f22d9d255cdb1987d25572f6ebcb37
+size 1346273240

model-00075-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:55992a5b643dad0a06c6bd8e4a13bac8980d31f3e773d56cc86a0c36543c02e5
+size 1346272736

model-00076-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3dea15fd98ef8c45a23876a3dd1f3f6c06feb6c5e3a35b29dde93a82196e9f63
+size 1570455164

model-00077-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fb6efae4033a54282fa0f779ce41dd64edff903621e829e23c28f2423f032059
+size 1346273240

model-00078-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1f231a1c8149307a14c940df734637e03b0cad5db40e4165c69b9a5e817471d9
+size 1346272424

model-00079-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:84eb7df2440c3fe0cad633bfdb05fc1f9f4a69514d33a3c48e08f751e8729b03
+size 1591901208

model-00080-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1008d2f310da3804fe85ca18c86f7f40c7e7005afe3edd0e17bff9207edb15cd
+size 1346273256

model-00081-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:30bb97e87009412ec4445773ab5dde2267f395290a37ea204b0b94fb78b06c16
+size 1591900360

model-00082-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6314d366c0a954e7a8beed9ead7389dc5c9ec6a78667f63778edb68a1ec5a5a7
+size 1346273240

model-00083-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:424127fa0a70e70d5073200c3ede1097dc31345b67309e9c1ca14fc1d2c601c9
+size 1346273096

model-00084-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8f5be1a45eeafd520fa3e0adceffb15cceaa6510c61d032475f017966885c248
+size 1591900536

model-00085-of-00120.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:22722c615db28a151620c8a6d11714f6de26bc386f3d495a89666191122ee1bf
+size 1346273256