Instructions to use Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4")
model = AutoModelForMultimodalLM.from_pretrained("Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4

SGLang

How to use Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4 with Docker Model Runner:
```
docker model run hf.co/Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4
```

Capicua25x commited on Aug 17

Commit

8dc02b8

verified ·

1 Parent(s): facf024

report GSM8K as a range: two runs exist at the same seed and the card published the higher one; align AA-LCR arm B to its score.json (0.800, not the rejudge pass) so every arm uses the same judging pass

Browse files

Files changed (1) hide show

README.md +15 -4

README.md CHANGED Viewed

@@ -140,12 +140,12 @@ on this same box, differing only in KV cache dtype.
 | benchmark | n | bf16 ref | FP8 + bf16 KV | FP8 + fp8 KV | **MXFP4 (this)** |
 |---|---|---|---|---|---|
-| GSM8K, thinking (flex / strict) | 50 | 0.96 / 0.82 | 0.96 / 0.90 | 0.94 / 0.70 | **0.96 / 0.94** |
 | GSM8K, no thinking (flex / strict) | 50 | 0.98 / 0.98 | 0.98 / 0.98 | 0.98 / 0.98 | **0.98 / 0.98** |
 | IFEval (inst / prompt, strict) | 80 | .9688 / .9500 | .9688 / .9500 | .9688 / .9500 | **.9688 / .9500** |
 | GPQA-Diamond (flexible) | 60 | 0.7833 | 0.8333 | 0.8333 | **0.9167** |
 | AIME 2025 | 30 | 0.9333 | 1.0000 | 0.9667 | **0.9333** |
-| AA-LCR (~107k-token prompts, judge-scored) | 100 | 0.780 | 0.800 ᵃ | 0.810 | **0.780** |
 | τ²-bench telecom (Pass^1) | 114 | 0.939 | 0.904 | 0.895 | **0.868** |
 | τ²-bench airline (Pass^1) | 50 | 0.760 | — | — | **0.840** |
 | HLE | 120 | 0.3083 | — | — | *running* |
@@ -155,6 +155,12 @@ on this same box, differing only in KV cache dtype.
 ᵃ Scored on the 90 items it served; 10 were refused because the prompt exceeded that
 configuration's 131k window. Blended over the full 100 it reads 0.720.
 **τ² is domain-split, and the split is the finding.** On telecom this build scores 0.868 against
 the bf16 reference's 0.939 — eight simulations — and sits four behind the FP8 + bf16 KV arm and
 three behind FP8 + fp8 KV. On airline it scores **0.840 against the reference's 0.760**, four items
@@ -165,8 +171,13 @@ On telecom, 113 of 114 simulations ended normally and one hit the harness's erro
 (`too_many_errors`, scored 0), so the shortfall is a genuine capability difference rather than
 harness noise — but it is one domain and a single-digit item count, not a blanket weakness.
-Everything else is at or above bf16: GSM8K strict-match **+6 items**, GPQA **+8 items**, and
-long-context retrieval **identical** to bf16 at ~107k-token prompts.
 Cells marked *running* / *pending* are genuinely unfinished, not withheld. This card is dated and
 will be revised as they land; the commit history is the record of what was known when.

 | benchmark | n | bf16 ref | FP8 + bf16 KV | FP8 + fp8 KV | **MXFP4 (this)** |
 |---|---|---|---|---|---|
+| GSM8K, thinking (flex / strict) | 50 | 0.96 / 0.82 | 0.96 / 0.90 | 0.94 / 0.70 | **0.94–0.96 / 0.92–0.94** ᶜ |
 | GSM8K, no thinking (flex / strict) | 50 | 0.98 / 0.98 | 0.98 / 0.98 | 0.98 / 0.98 | **0.98 / 0.98** |
 | IFEval (inst / prompt, strict) | 80 | .9688 / .9500 | .9688 / .9500 | .9688 / .9500 | **.9688 / .9500** |
 | GPQA-Diamond (flexible) | 60 | 0.7833 | 0.8333 | 0.8333 | **0.9167** |
 | AIME 2025 | 30 | 0.9333 | 1.0000 | 0.9667 | **0.9333** |
+| AA-LCR (~107k-token prompts, judge-scored) | 100 | 0.780 | 0.800 ᵃ | 0.800 | **0.780** |
 | τ²-bench telecom (Pass^1) | 114 | 0.939 | 0.904 | 0.895 | **0.868** |
 | τ²-bench airline (Pass^1) | 50 | 0.760 | — | — | **0.840** |
 | HLE | 120 | 0.3083 | — | — | *running* |
 ᵃ Scored on the 90 items it served; 10 were refused because the prompt exceeded that
 configuration's 131k window. Blended over the full 100 it reads 0.720.
+ᶜ **Two runs of this build exist at the same seed and identical settings** — 0.94/0.92 and
+0.96/0.94 — so the honest figure is a range, not a point. The other three columns are single
+runs, which is worth knowing before reading small deltas here as real: on this cell one run's
+difference is one item. Against the reference's 0.82 strict, this build is +5 or +6 items
+depending on which run you take.
 **τ² is domain-split, and the split is the finding.** On telecom this build scores 0.868 against
 the bf16 reference's 0.939 — eight simulations — and sits four behind the FP8 + bf16 KV arm and
 three behind FP8 + fp8 KV. On airline it scores **0.840 against the reference's 0.760**, four items
 (`too_many_errors`, scored 0), so the shortfall is a genuine capability difference rather than
 harness noise — but it is one domain and a single-digit item count, not a blanket weakness.
+Everything else is at or above bf16: GSM8K strict-match **+5 to +6 items** (see note ᶜ — two
+runs exist), GPQA **+8 items**, and long-context retrieval **identical** to bf16 at ~107k-token
+prompts.
+All AA-LCR figures are the runner's own judging pass, taken from each arm's `score.json`. A
+second judging pass over the same generations moves scores by roughly one item in either
+direction; mixing passes between arms would manufacture differences that are not there.
 Cells marked *running* / *pending* are genuinely unfinished, not withheld. This card is dated and
 will be revised as they land; the commit history is the record of what was known when.