---
license: apache-2.0
base_model: amd/Qwen3.8-27B-Quark-Qronos-INT4-W4A16
base_model_relation: quantized
library_name: paiton
pipeline_tag: text-generation
inference: false
tags:
  - paiton
  - amd
  - rocm
  - rdna4
  - gfx1201
  - compiled-model
---

# Paiton Qwen3.8 27B on AMD Radeon AI PRO R9700

Run [amd/Qwen3.8-27B-Quark-Qronos-INT4-W4A16](https://huggingface.co/amd/Qwen3.8-27B-Quark-Qronos-INT4-W4A16) locally through
Paiton's compiled AMD GPU runtime and an OpenAI-compatible API.

This repository contains **compiled runtime artifacts, configuration and
manifests only**, about **7.21 MB**. Model weights are downloaded
directly from the original publisher and remain in your local cache.
The `.so` files contain executable model graphs and GPU kernels. The private
Paiton compiler is not required to use them and is not included.

**Release:** v1.3.0 · **GPU:** Radeon AI PRO R9700 (`gfx1201`, 32 GB)
· **Batch:** 1 · **Tensor parallelism:** 1 · **Context:** 8,192 tokens · **Text only**

## Quick start

You need Linux x86-64, Docker and working ROCm device access through
`/dev/kfd` and `/dev/dri`. The image includes the qualified ROCm 7.14,
PyTorch, vLLM and Paiton runtime. This release is qualified on the R9700;
other GPUs and runtime versions are not covered by that qualification.

```bash
docker run -d \
  --name paiton-qwen38 \
  --device /dev/kfd \
  --device /dev/dri \
  --group-add video \
  --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v paiton-qwen38-cache:/models/cache \
  ghcr.io/eliovp/paiton-vllm-plugin:qwen38-qronos-rdna4-v1.3.0
```

The first start downloads the 19.9 GB checkpoint directly from AMD's original Hugging Face repository. The named volume caches it for later runs. With cached weights, the published guide reports roughly 10–12 minutes for model assembly on the qualified host. Wait for readiness before sending requests.

The container already bundles the same compiled artifacts published here.
You do not need to download the `.so` separately for this quick start.
The upstream repositories are public; no Hugging Face token is required.
Run one example server at a time because both use port 8000.

Follow startup:

```bash
docker logs -f paiton-qwen38
```

Wait for `Application startup complete.` Then, from another terminal:

```bash
curl --fail http://127.0.0.1:8000/v1/models
docker exec -it paiton-qwen38 paiton-chat --model qwen38
```

Use `/reset` to clear the chat and `/quit` to leave it. Or call the API:

```bash
curl --fail http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen38","messages":[{"role":"user","content":"Explain why the sky is blue."}],"temperature":0,"max_tokens":256,"chat_template_kwargs":{"enable_thinking":false}}'
```

## Model and execution details

The Hub quantization relationship identifies this compiled runtime package
with its upstream quantized checkpoint. It does not indicate that this
repository publishes a new set of quantized weights.

- Original repository: [amd/Qwen3.8-27B-Quark-Qronos-INT4-W4A16](https://huggingface.co/amd/Qwen3.8-27B-Quark-Qronos-INT4-W4A16).
- Pinned weight revision: `649ca9d47a7de5364c6fcccc0c1b4f6e542e15e2`.
- Execution format: Qronos INT4 checkpoint; fitted INT3 MLP decode shadows and W4 LM head.
- One active request; additional requests queue. Changing server flags does
  not add multi-batch or multi-GPU support to this release.

This release uses the original Qronos W4 checkpoint for prefill and fitted decode-only INT3 MLP shadows during generation. The runtime prepares these shadows locally; this repository contains no fitted weight tensors. Decode quantization intentionally differs from source W4 execution, so bit-identical output to stock W4 is not promised. Thinking is disabled by default.

## Performance and quality

This Hub package redistributes the existing v1.3.0 release without
changing its artifacts. It does not introduce a new benchmark or claim
task-quality equivalence with stock W4 execution. See the
[published release](https://github.com/Eliovp-BV/paiton-vllm-plugin/releases/tag/paiton-qwen38-qronos-w4a16-gfx1201-v1.3.0)
and [runtime guide](https://github.com/Eliovp-BV/paiton-vllm-plugin/blob/58b0d0b31a48768e08378c6b999ca49a6ac5b398/models/Qwen3.8/README.md) for the released configuration.


## Download and use these Hub artifacts

For manual artifact use, install the Hugging Face CLI in your local Python
environment, then download the release into a separate directory:

```bash
python3 -m pip install huggingface_hub
hf download EliovpAI/Qwen3.8-27B-Quark-Qronos-INT4-W4A16-Paiton-RDNA4 \
  --revision v1.3.0 --include 'overlay/*' \
  --local-dir ./paiton-qwen38-hf
```

Use the quick-start Docker command above, adding these two options **before
the image name**:

```bash
  -e PAITON_OVERLAY_DIR=/models/paiton-overlay \
  -v "$PWD/paiton-qwen38-hf/overlay:/models/paiton-overlay:ro" \
```

The runtime validates the artifact hashes before use. Keep the `overlay/`
directory intact: it contains the exact files expected by the release loader.
The `.so` is loaded by Paiton; it is not a standalone Transformers checkpoint.

For an immutable runtime reference, replace the friendly image tag with:

```text
ghcr.io/eliovp/paiton-vllm-plugin@sha256:c56baf54aca1ad229829c1de26e8792806608e65ee9d06ee210b79cd49f70bc9
```

File hashes and provenance are in [SHA256SUMS](SHA256SUMS) and
[paiton-hub.json](paiton-hub.json). All artifacts are byte-for-byte copies
of the published release. This repository contains no checkpoint weights,
draft weights, calibration data or compiler sources.

## Links and licenses

- [Public Paiton plugin](https://github.com/Eliovp-BV/paiton-vllm-plugin) and [complete model guide](https://github.com/Eliovp-BV/paiton-vllm-plugin/blob/58b0d0b31a48768e08378c6b999ca49a6ac5b398/models/Qwen3.8/README.md).
- [Paiton](https://eliovp.com/products/paiton).
- [Our other RDNA4 model: Ornith-1.5](https://huggingface.co/EliovpAI/Ornith-1.5-35B-A3B-MXFP4-Paiton-RDNA4).

The Paiton release is distributed under [Apache-2.0](LICENSE), with retained
[third-party notices](THIRD_PARTY_NOTICES.md) and [license texts](LICENSES/).
Original model and draft terms remain those of their respective upstream
repositories. This artifact package does not change those terms.