---
license: other
license_name: qwen-research
license_link: https://huggingface.co/Qwen/Qwen-Image-2.1/blob/main/LICENSE
base_model: Qwen/Qwen-Image-2.1
tags: [text-to-image, image-to-image, quantization, w4a4, svdquant, gptq, nunchaku]
pipeline_tag: text-to-image
---

# Qwen-Image-2.1 W4A4 (nvfp4)

Built with Qwen.

> **Non-commercial use only.** An educational artifact under the Qwen Research License,
> which the base model carries. Commercial use needs a separate licence from the rights
> holder; publishing this here does not grant one.

4-bit weights, 4-bit activations, for the Qwen-Image-2.1 transformer. Calibrated with
SVDQuant, error-compensated with GPTQ, packed for Nunchaku's fused kernel.
**4.54 GB** against about 10 GB for the same layers in bf16.

The low-rank branch is fused into the 4-bit GEMM, which is where the speed comes from —
an unfused branch gives back 24–34% of it.

## Hardware

**Blackwell only (sm_120 and up).** This will not load on Ada — use the int4 build.

## Measured

| card | cap | seconds | vs bf16 | resident GB | LPIPS vs bf16 |
|---|---|---|---|---|---|
| RTX PRO 6000 Max-Q | sm_120 | 7.747 | 1.71x | 21.53 | 0.1425 |
| RTX 5090 | sm_120 | 7.653 | — | 21.53 | — |

40 steps, 1024×1024, seed 42, six prompts. LPIPS is AlexNet against the bf16 image of
the same prompt on the same card.

**GPTQ is worth 23% here**:
0.1858 with round-to-nearest against **0.1425** with GPTQ, at the same
speed, the same file size and the same resident memory — it only changes which grid point
each weight lands on.

Read LPIPS as distance from the bf16 trajectory, not as damage. On a dense-text prompt
every build renders the text correctly while scoring 0.24–0.35, and on one prompt the
bf16 reference garbles a sign that this checkpoint renders correctly — and is scored
worse for it.

Only the transformer is quantized. The Qwen3-VL text encoder stays bf16 and is about
16.3 GB of a 21.5 GB resident total, so it, not this file, decides how many reference
images fit on a card.

## Text accuracy against bf16

Every LPIPS number above is measured *against* bf16, so it cannot answer "is this as good
as bf16" — it defines bf16 as zero. This is the reference-free version: 8 seeds on each of
two text prompts, scored by reading the rendered text against what the prompt asked for.

| build | enamel notice board | neon shop sign | total |
|---|---|---|---|
| bf16 | 8/8 | 6/8 | **14/16** |
| this checkpoint | 8/8 | 7/8 | **15/16** |

**Indistinguishable.** One render's difference out of sixteen, which a different seed set
would flip. The errors trade places rather than accumulating on one side: on one seed bf16
renders the sign as "QWEN 2.1 IMAGE 1" and this checkpoint gets it right; on another this
checkpoint appends a spurious "1$" where bf16 is clean.

That is the claim worth making — **4-bit weights and activations with no measurable text
degradation** — and not the stronger one. A single prompt where the quantized build beats
bf16 is trajectory variance, not evidence of superiority.

An automated OCR pass was tried first and discarded: tesseract returns noise on neon at
night, and a character-error-rate above 1.0 was the giveaway that it was scoring incidental
scene text rather than the sign. The table above comes from reading all 48 renders.

`samples/` holds one render per prompt at seed 42. `seeds/` holds the full sweep.

## Requirements

Packed for a specific kernel. The layout is an MMA fragment swizzle, so a different
Nunchaku build may read it incorrectly — and would produce a wrong image rather than an
error. The loader checks and warns.

```
torch==2.12.1
nunchaku==1.3.0.dev20260306+cu13.0torch2.12
```

## Loading

```python
import torch
from diffusers import QwenImage21Pipeline
from quantize import nunchaku_io

transformer = nunchaku_io.load_checkpoint(precision="nvfp4", device="cuda")
pipe = QwenImage21Pipeline.from_pretrained(
    "Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16, transformer=transformer)
```

`nunchaku_io` and the build pipeline: https://github.com/ModelsLab/qwen-image-2-1-server

## How it was built

Smoothing factors from real activation statistics (SmoothQuant, α=0.5), a rank-128 SVD
branch kept in bf16, and the residual rounded onto Nunchaku's grid by GPTQ against a
calibration Hessian. One Hessian pass, 128 distinct inputs across 224 layers, 24 GB
resident, ~5 minutes.

`w4a4_build.json` records the whole build, including per-layer GPTQ statistics.

## License

Derivative of [Qwen/Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1) under the
**Qwen Research License Agreement** — non-commercial only. See `NOTICE`.