---
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/LICENSE
base_model: Qwen/Qwen3.6-27B
base_model_relation: quantized
pipeline_tag: text-generation
quantized_by: 1337Hero
tags:
  - gguf
  - qwen3.6
  - quantized
  - rocm
  - amd
  - rdna4
  - gfx1201
  - experimental
---

# Qwen3.6-27B — Q8_0_ROCMFPX GGUF (experimental, AMD gfx1201)

An experimental Q8 quantization of
[Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B), tuned and
validated for the AMD Radeon AI PRO R9700 (`gfx1201`).

> [!IMPORTANT]
> This file does **not** run on upstream llama.cpp, Ollama, LM Studio, or
> vLLM. It uses the custom `Q8_0_ROCMFPX` tensor type and requires the pinned
> ROCmFPX fork build described below. Unsupported runtimes should reject the
> file; if a tool appears to load it anyway, do not trust the output. Hugging
> Face's GGUF metadata viewer may also mislabel the custom tensor type or fail
> to parse the file.

## Should you use this?

Use this model if all of the following are true:

- you run a Radeon AI PRO R9700 (`gfx1201`) under ROCm;
- you are willing to build the pinned fork from source;
- you want a Q8 that is 2.94% smaller than upstream `Q8_0` and performs equal
  observed model work about 6% faster on this hardware.

Otherwise, use an ordinary `Q8_0` GGUF of Qwen3.6-27B. It has broad support in
the standard GGUF runtime ecosystem, and its quality is equivalent within the
tolerances measured here.

## TL;DR — measured against upstream `Q8_0`

| Measure | Result |
| --- | ---: |
| Model size | 2.94% smaller |
| HumanEval | 140/164 — tied |
| MBPP base / MBPP+ | 348/378 and 291/378 vs 350/378 and 293/378 — within the predeclared tolerance |
| Proxy — full-model decode | 5.77–5.87% faster |
| Proxy — `pp4096+tg512` combined | 4.75–4.96% faster |
| Equal-work — agent-derived evaluation (18 matched pairs) | 5.82% faster median, 17/18 pairs, exact one-sided `p=0.000965` |
| End-to-end — raw live-agent wall time | 2.21% faster — **failed** the predeclared 3% threshold |

The supported conclusion is deliberately narrow: this format performs equal
observed model work faster than upstream `Q8_0` on the measured deployment.
The campaign did **not** confirm that complete live-agent jobs finish at
least 3% faster, so no such claim is made. All thresholds were fixed before
the runs; failed gates are reported, not reinterpreted.

The public [claim registry](claims.yaml) records the scope, result class, and
sealed aggregate-evidence hash behind each release claim.

## What `Q8_0_ROCMFPX` actually is

Each block stores 32 signed eight-bit codes and one finite UE4M3 scale byte:
33 bytes per 32 weights, or 8.25 bits per weight. The HIP path decodes the
scale and performs integer MMVQ/MMQ dot products with float accumulation.
This is **not** native FP8 or FP4 matrix arithmetic.

```text
GGUF block
  32 signed int8 codes + UE4M3 scale
        |
        v
activation quantization to Q8_1
        |
        +--> decode:  gfx1201 MMVQ, VDR8 + measured wave policy
        |
        `--> prefill: integer MMQ path
        |
        v
float accumulation / model output
```

The experimental format and execution path come from the upstream
[ROCmFPX](https://github.com/charlie12345/ROCmFPX) project. This repository's
contribution is the `gfx1201` decode tuning (VDR8 vector-dot width and a
measured wave policy) and the validation below; other GPU targets retain
ROCmFPX defaults and are unmeasured.

The quantization is uniform Q8 with no importance matrix and no per-tensor
routing.

## Required runtime

Build the pinned fork branch:

```bash
git clone https://github.com/1337hero/ROCmFPX.git
cd ROCmFPX
git checkout 45bcff509c4b1cff137e2cc1ea84671c61ceddea
env JOBS=16 scripts/build-r9700.sh
```

The wrapper is the simplest supported build. The sealed performance runners
used this HIP-only configuration:

```bash
cmake -S . -B build-r9700 -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DGGML_HIP=ON \
  -DGGML_HIP_FORCE_MMQ=ON \
  -DGGML_VULKAN=OFF \
  -DGGML_CUDA=OFF \
  -DCMAKE_HIP_ARCHITECTURES=gfx1201 \
  -DGPU_TARGETS=gfx1201 \
  -DCMAKE_HIP_FLAGS= \
  -DLLAMA_BUILD_SERVER=ON \
  -DLLAMA_BUILD_WEBUI=OFF \
  -DLLAMA_USE_PREBUILT_WEBUI=OFF \
  -DLLAMA_BUILD_TESTS=ON \
  -DGGML_BUILD_TESTS=OFF
cmake --build build-r9700 -j 16 --target \
  llama-server llama-bench llama-quantize
```

The validated implementation is ROCmFPX base commit
`6bf20cd688ba0af882d1f68ba50b292edf646ab4` plus commits
`eb38c6f67701ff9c74e8597f573eedf9ccecf774` and
`45bcff509c4b1cff137e2cc1ea84671c61ceddea`.

Binaries validated by the lab:

| Binary | SHA-256 |
| --- | --- |
| `llama-server` | `1471753a94ba007b094842474cc3b3ffd48106f15a7c71934899dd832ae4cbb3` |
| `llama-bench` | `38bc30e53badc5ae5efb5e3d449d989a4672c52fe5801eb1ce74e70233283749` |
| `llama-quantize` | `0aa545af25e8235613349987fb29323745b70980aa2a4fb45c169cdf561181c6` |

## Example deployment

This shape matches the validated two-card, 262,144-token envelope. Device
names depend on the host; check `--list-devices` first.

```bash
./build-r9700/bin/llama-server \
  -m Qwen3.6-27B-Q8_0_ROCMFPX.gguf \
  --no-mmap \
  -c 262144 \
  -b 2048 \
  -ub 512 \
  -t 16 \
  -ngl 99 \
  -sm layer \
  -ts 1,1 \
  -dev ROCm0,ROCm2 \
  --fit off \
  -ctk f16 \
  -ctv f16 \
  -fa on \
  -np 1
```

Both this model and the upstream `Q8_0` control allocated one 262,144-token
slot across two R9700s with F16 K/V cache and generated bounded valid output;
this model used 804,319,232 fewer resident bytes in the captured snapshots.
That test proved allocation and bounded generation, not a timed full-context
prompt.

## Measurement conditions

All performance numbers were measured on:

- Qwen3.6-27B, this GGUF vs an upstream `Q8_0` control of the same
  conversion, same host, same workload;
- two non-display Radeon AI PRO R9700s at a pre-existing 250 W, −75 mV tune —
  **not** stock 300 W;
- ROCm 7.2.4 on Arch Linux, outside the official Radeon Linux support matrix;
- pinned source revisions, forced model residency, reversed candidate order,
  and predeclared promotion thresholds.

The equal-work result comes from 18 common-seed matched agent-workload pairs
with ABBA/BAAB counterbalancing, warmup before every measured run, full
request accounting (1,899 logical requests, zero retries), and exact paired
Wilcoxon signed-rank tests.

## Artifact

| Field | Value |
| --- | --- |
| File | `Qwen3.6-27B-Q8_0_ROCMFPX.gguf` |
| Size | 27,755,345,216 bytes |
| SHA-256 | `ff4dbc9093c1df6fd1242294d15eb94c7bfe42ed67f98e9d58b9789ac6912c1b` |
| Source GGUF | BF16, SHA-256 `0438be1f5bc861ffa84e1d2d4036920f6f3d9759f3cdedbc40e554a321d1c9c5` |
| Quantization | uniform `Q8_0_ROCMFPX`, no importance matrix |
| Content | main text model only; no MTP artifact or multimodal projector |

Quantization command (the qualifying run wrote to a temporary filename and
then promoted the verified hash without overwriting an existing artifact):

```bash
./build-r9700/bin/llama-quantize \
  Qwen3.6-27B-BF16.gguf \
  Qwen3.6-27B-Q8_0_ROCMFPX.gguf \
  Q8_0_ROCMFPX \
  16
```

Verify after download:

```bash
sha256sum -c SHA256SUMS
```

## Limitations

- Requires an experimental fork; upstream llama.cpp compatibility awaits a
  separate code contribution.
- Validated on exactly one model conversion, one Arch Linux host, ROCm 7.2.4,
  and `gfx1201` GPUs at a tuned power profile. Other GPUs, hosts, and stock
  power are unmeasured.
- Aggregate quality stays within the declared tolerances, but this model and
  upstream `Q8_0` do not produce identical per-task outcomes.
- The raw private lab archive (prompts, outputs, paths, environment) is not
  published; it awaits a separate sanitization review.

## License and attribution

- **Base model:** Qwen3.6-27B, Copyright 2026 Alibaba Cloud, Apache-2.0. This
  repository redistributes a converted and quantized derivative under the same
  license. See `LICENSE` and `NOTICE`.
- **Format and execution path:** the experimental `Q8_0_ROCMFPX`
  representation and HIP kernels are the work of the
  [ROCmFPX](https://github.com/charlie12345/ROCmFPX) project, which builds on
  [llama.cpp](https://github.com/ggml-org/llama.cpp).
- **This repository:** the `gfx1201` decode tuning, quantized artifact, and
  validation evidence.

Qwen and related marks belong to their owners. This community quantization is
not affiliated with or endorsed by Qwen, Alibaba Cloud, AMD, ROCmFPX, or
llama.cpp.