File size: 3,699 Bytes
117b828 141d236 117b828 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 | ---
license: other
license_name: qwen-research
license_link: https://huggingface.co/Qwen/Qwen-Image-2.1/blob/main/LICENSE
base_model: Qwen/Qwen-Image-2.1
tags: [text-to-image, image-to-image, quantization, w4a4, svdquant, gptq, nunchaku]
pipeline_tag: text-to-image
---
# Qwen-Image-2.1 W4A4 (int4)
Built with Qwen.
> **Non-commercial use only.** An educational artifact under the Qwen Research License,
> which the base model carries. Commercial use needs a separate licence from the rights
> holder; publishing this here does not grant one.
4-bit weights, 4-bit activations, for the Qwen-Image-2.1 transformer. Calibrated with
SVDQuant, error-compensated with GPTQ, packed for Nunchaku's fused kernel.
**4.34 GB** against about 10 GB for the same layers in bf16.
The low-rank branch is fused into the 4-bit GEMM, which is where the speed comes from —
an unfused branch gives back 24–34% of it.
## Hardware
**sm_75 through sm_120** — Ada (4090, L40S) and Blackwell alike.
## Measured
| card | cap | seconds | vs bf16 | resident GB | LPIPS vs bf16 |
|---|---|---|---|---|---|
| RTX 4090 48GB | sm_89 | 11.576 | 1.84x | 21.35 | 0.231 |
| RTX PRO 6000 Max-Q | sm_120 | 37.616 | 0.35x | 21.29 | 0.1836 |
40 steps, 1024×1024, seed 42, six prompts. LPIPS is AlexNet against the bf16 image of
the same prompt on the same card.
**GPTQ is worth 6% here**:
0.1946 with round-to-nearest against **0.1836** with GPTQ, at the same
speed, the same file size and the same resident memory — it only changes which grid point
each weight lands on.
Read LPIPS as distance from the bf16 trajectory, not as damage. On a dense-text prompt
every build renders the text correctly while scoring 0.24–0.35, and on one prompt the
bf16 reference garbles a sign that this checkpoint renders correctly — and is scored
worse for it.
Only the transformer is quantized. The Qwen3-VL text encoder stays bf16 and is about
16.3 GB of a 21.5 GB resident total, so it, not this file, decides how many reference
images fit on a card.
## Text accuracy against bf16
The nvfp4 build of this same recipe scored 15/16 against bf16's 14/16 on a reference-free seed sweep (8 seeds, two text prompts, text read against the prompt) — indistinguishable. This int4 build was not put through that sweep; its LPIPS is 0.1836 against the nvfp4 build's 0.1425, so treat it as the same recipe at slightly lower fidelity, on the far wider set of cards it runs on.
## Requirements
Packed for a specific kernel. The layout is an MMA fragment swizzle, so a different
Nunchaku build may read it incorrectly — and would produce a wrong image rather than an
error. The loader checks and warns.
```
torch==2.12.1
nunchaku==1.3.0.dev20260306+cu13.0torch2.12
```
## Loading
```python
import torch
from diffusers import QwenImage21Pipeline
from quantize import nunchaku_io
transformer = nunchaku_io.load_checkpoint(precision="int4", device="cuda")
pipe = QwenImage21Pipeline.from_pretrained(
"Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16, transformer=transformer)
```
`nunchaku_io` and the build pipeline: https://github.com/ModelsLab/qwen-image-2-1-server
## How it was built
Smoothing factors from real activation statistics (SmoothQuant, α=0.5), a rank-128 SVD
branch kept in bf16, and the residual rounded onto Nunchaku's grid by GPTQ against a
calibration Hessian. One Hessian pass, 128 distinct inputs across 224 layers, 24 GB
resident, ~5 minutes.
`w4a4_build.json` records the whole build, including per-layer GPTQ statistics.
## License
Derivative of [Qwen/Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1) under the
**Qwen Research License Agreement** — non-commercial only. See `NOTICE`.
|