---
license: other
license_name: qwen-research-license
license_link: LICENSE
base_model: Qwen/Qwen-Image-2.1
base_model_relation: quantized
pipeline_tag: text-to-image
library_name: diffusers
tags:
  - qwen-image
  - qwen-image-2.1
  - nvfp4
  - svdquant
  - nunchaku
  - blackwell
  - text-to-image
  - image-editing
---

# Qwen-Image-2.1-NVFP4 (SVDQuant, DiT + text encoder, native in Diffusers)

**Built with Qwen.** Non-commercial research and evaluation use only (Qwen Research License, see `LICENSE` and `NOTICE`).

An NVFP4 quantization of [Qwen/Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1), made
from the official BF16 weights with SVDQuant + GPTQ and calibrated on both text-to-image prompts and
image edits. Both large components run on native Blackwell FP4 tensor cores, and the pipeline loads with a
single `DiffusionPipeline.from_pretrained` call.

At 512x512 on an RTX PRO 6000 Blackwell it is about **1.7x faster than BF16 and needs about 40% less VRAM**,
and it matches BF16 closely on text-to-image, typography and edits, including a hard recolor edit that
earlier 4-bit text encoders failed.

## Usage

```python
import torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "joseplcam/Qwen-Image-2.1-NVFP4", dtype=torch.bfloat16, trust_remote_code=True
).to("cuda")

image = pipe("A capybara reading a book by candlelight", height=512, width=512, num_inference_steps=40).images[0]
edited = pipe("Make it night time with moonlight", image=image, output_resolution=512, num_inference_steps=40).images[0]
```

`trust_remote_code=True` loads two small files from this repository:

- `text_encoder/modeling_nunchaku_qwen3vl.py`: a `Qwen3VLForConditionalGeneration` subclass that swaps the
  quantized linears for Diffusers' own `SVDQW4A4Linear` before loading the weights.
- `transformer/modeling_nunchaku_qwenimage21.py`: the stock transformer class, unchanged.

Both files start with the same kernel setup, so it runs whichever component loads first. Diffusers loads
the NVFP4 kernels from `rootonchair/nunchaku-lite-kernels`, which is no longer downloadable. The setup
points that name at [joseplcam/nunchaku-lite-kernels](https://huggingface.co/joseplcam/nunchaku-lite-kernels),
an unmodified build of the same open-source kernels, and sets `DIFFUSERS_TRUST_REMOTE_KERNELS=true` unless
you already set it. Set `LOCAL_KERNELS` yourself to use a different build.

Use a different seed for an edit than the one that generated its input image. Qwen-Image-2.1 returns an
over-sharpened copy that ignores the prompt when the edit starts from the same noise
([diffusers #14824](https://github.com/huggingface/diffusers/issues/14824)); this is a base-model behaviour.

### Requirements

- NVIDIA Blackwell GPU with compute capability 12.0 (RTX 50 series, RTX PRO 6000). The published kernel build
  targets `sm_120a` only.
- Linux x86_64, Python 3.12, PyTorch 2.13 with CUDA 13.0, which the kernel build targets.
- Diffusers from `main` with Qwen-Image-2.1 and Nunchaku Lite support (tested at commit `0377f0c`),
  `transformers>=5.12`, `accelerate`, `kernels>=0.14`.

## What is inside

| Component | Precision | Details |
|---|---|---|
| `transformer/` (7B DiT) | NVFP4 W4A4, group 16, FP8 block scales + BF16 rank-32 low-rank branch | All attention and MLP projections of blocks 2-29 (196 layers). Blocks 0, 1, 30, 31 and the global modulation stay BF16. |
| `text_encoder/` (Qwen3-VL 8B) | NVFP4 W4A4 + BF16 rank-128 low-rank branch | MLP projections (gate/up/down) of decoder layers 4-31 (84 layers). All attention projections, layers 0-3 and 32-35, the vision tower, embeddings and `lm_head` stay BF16. |
| `vae/` | BF16 | Official weights cast from FP32. |
| `processor/`, `scheduler/` | as released | Unmodified. |

Total download: about 17 GB, against about 32 GB for the BF16 release.

## How it was made

- **Method:** SVDQuant with GPTQ residual rounding, via [diffuse-compressor](https://github.com/rootonchair/diffuse-compressor)
  (commit `0965874`). A low-rank BF16 branch absorbs the outliers of each weight and SmoothQuant-style
  scaling migrates activation outliers; GPTQ then rounds the 4-bit residual using calibration statistics.
  Activation scales are dynamic, so nothing is fixed to the calibration inputs.
- **Calibration data (128 samples per component):** 64 prompts from the qdiff prompt set and 64 image edits
  from the train split of [VyoJ/NHR-Edit-Change_Only](https://huggingface.co/datasets/VyoJ/NHR-Edit-Change_Only).
  Three of every four samples at 512x512, the rest at 1024x1024. The DiT saw 20 denoising steps per sample,
  with the prefix KV cache disabled so the prompt and reference-image tokens pass through every step.
- **Sensitive layers:** the first and last two DiT blocks and the modulation stay BF16. For the text encoder,
  quantizing attention hurt edits most. With every encoder linear quantized, the "turn the parrots blue"
  recolor below worked in 1 of 8 seeds (rank 32) or 4-5 of 8 (rank 128); quantizing only the MLPs with rank
  128 brought it to 7 of 8, the same as BF16.
- **Runtime:** Diffusers' built-in Nunchaku Lite quantizer for the DiT and the same `SVDQW4A4Linear` layers
  for the text encoder, running the [nunchaku-lite](https://github.com/rootonchair/nunchaku-lite) CUDA kernels.

The scripts that produced this repository are in `tools/`: `quantize_dit.py`,
`quantize_text_encoder.py --rank 128 --edge-layers 4 --skip-attention`, `package.py`, `evaluate.py` and
`parrot_test.py`. They need a checkout of diffuse-compressor (its `examples/` package) at
`$DIFFUSE_COMPRESSOR`.

## Results

RTX PRO 6000 Blackwell (96 GB), Diffusers, 40 steps, guidance 1, no `torch.compile`, after a warmup.
Peak VRAM is PyTorch's peak allocation.

| | BF16 (official) | This repo |
|---|---|---|
| Text-to-image, 512x512 | 3.32 s | **1.91 s** (1.74x) |
| Edit, 512x512 | 3.73 s | **2.18 s** (1.71x) |
| Text-to-image, 1024x1024 | 14.06 s | **8.12 s** (1.73x) |
| Peak VRAM, 512x512 | 31.9-32.5 GiB | **18.6-19.3 GiB** |
| Throughput, 512x512 batch 1-8 | 0.28-0.30 img/s | **0.49-0.55 img/s** |

Batching several prompts gives no extra throughput in Diffusers on this GPU; it is saturated at batch 1.

Runtime options (text-to-image, 512x512, 40 steps, one fixed prompt, mean of 3 runs after warmup):

| Setup | BF16 (official) | This repo |
|---|---|---|
| Eager, PyTorch SDPA | 3.20 s | 1.79 s |
| Eager, cuDNN SDPA | 3.26 s | 1.88 s |
| `compile_repeated_blocks()`, PyTorch SDPA | 2.99 s | 1.50 s |

- **Use eager (the default) for mixed workloads.** `torch.compile` only wins while every call has the same
  shapes. Prompt length and the reference image's aspect ratio change the prefix KV cache, so varied
  text-to-image prompts and edits keep recompiling until TorchDynamo's recompile limit, after which new shapes
  run uncompiled anyway. In a 12-call mix (6 prompts of different lengths, 6 edits with different reference
  shapes) eager took 37.2 s in total, `compile_repeated_blocks()` 44.3 s and `dynamic=True` 56.5 s; steady-state
  edits were 2.0 s, 2.0 s and 1.7 s. Compile is only worth it for fixed-shape batch jobs.
- CUDA graphs (`mode="reduce-overhead"`) do not work: the prefix KV cache keeps tensors that graph replays
  overwrite, and the Nunchaku kernels cannot be captured (`cudaErrorStreamCaptureInvalidated`). With the KV
  cache off, BF16 gained only 6% from them.
- SageAttention and PyTorch's flash SDPA reject the attention mask Qwen-Image-2.1 passes, so they cannot be
  used with this model in Diffusers.
The pipeline shares one reference-image list across a batch, so edits with different images run one at a time.

Closeness to BF16 (LPIPS with AlexNet, same seeds, 512x512; lower is closer, around 0.1 is hard to tell apart):

| Set | LPIPS vs BF16 |
|---|---|
| 8 text-to-image prompts (portraits, typography, counting, scenes) | 0.148 |
| 12 held-out NHR-Edit test edits + 1 parrot recolor | 0.046 |

Instruction following on a hard recolor ("Turn the parrots blue", 8 seeds, tiles with at least one blue parrot),
with this repo's DiT:

| Text encoder | Followed |
|---|---|
| BF16 | 7 / 8 |
| NVFP4, every linear, rank 32 | 1 / 8 |
| NVFP4, every linear, rank 128, 4 BF16 edge layers | 4-5 / 8 |
| **NVFP4, MLPs only, rank 128, 4 BF16 edge layers (this repo)** | **7 / 8** |

With the BF16 text encoder, this repo's DiT followed the edit in 8 of 8 seeds, the same as the BF16 DiT.

## Limitations

- Measured on one GPU model and one software stack; quantization shifts details of individual images.
- Not bit-reproducible: the same seed gives slightly different images from run to run (LPIPS 0.04-0.17
  between repeats, against about 0.001 for BF16), most likely from non-deterministic accumulation in the
  W4A4 kernels. Part of the LPIPS against BF16 above is this variation.
- The evaluation is small (8 prompts, 13 edits, one 8-seed recolor test). It is not a benchmark.
- The kernel build covers `sm_120a`, PyTorch 2.13, CUDA 13 and CPython 3.12 only.
- Non-commercial research and evaluation use only.

## Previous version

Until 2026-09-25 this repository held a different combination for SGLang Diffusion: the ModelOpt NVFP4 DiT from
HangGlidersRule/Darkstar-Qwen-Image-2.1-Base-ModelOpt-W4A4-NVFP4 and the NVFP4 Qwen3-VL encoder from
BennyDaBall/Qwen-Image-2.1-NVFP4. It is still available in this repository's commit history. It was
replaced because its 4-bit text encoder lost hard edits and it did not load in Diffusers.

## Credits and license

- Qwen team: Qwen-Image-2.1.
- [SVDQuant](https://arxiv.org/abs/2411.05007) (MIT Han Lab), [nunchaku-lite](https://github.com/rootonchair/nunchaku-lite)
  and [diffuse-compressor](https://github.com/rootonchair/diffuse-compressor) (rootonchair).
- Calibration edits: [VyoJ/NHR-Edit-Change_Only](https://huggingface.co/datasets/VyoJ/NHR-Edit-Change_Only).

Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) 2026 Hangzhou Tongyi Laboratory
Technology Co., Ltd. All Rights Reserved. See `LICENSE` and `NOTICE`, which lists every modified file.