---
license: apache-2.0
base_model: Qwen/Qwen-Image-2.1
library_name: nunchaku
tags:
- qwen
- qwen-image
- nunchaku
- svdquant
- nvfp4
- text-to-image
- image-to-image
- diffusers
---

# Qwen-Image-2.1 (Nunchaku SVDQuant NVFP4 r32 with Channel-Wise Macro Scaling) 🌟🚀

This repository provides the high-fidelity **SVDQuant NVFP4 (Rank 32)** quantized transformer for Alibaba's **[Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1)**, forged with **channel-wise Level-0 macro scaling (`wcscales`)** for inference with **[Nunchaku](https://github.com/mit-han-lab/nunchaku)** on NVIDIA Blackwell (`sm_120`), Ada Lovelace, and Hopper architectures.

---

## ⚡ Highlights

* **100% Resident VRAM Execution on 16GB GPUs**:
  * Weights compressed from **30.2 GB (BF16)** down to **4.08 GB (NVFP4)** — a **7.4x reduction** in transformer memory footprint!
  * Runs entirely resident in VRAM on consumer GPUs (e.g. NVIDIA GeForce RTX 5060 Ti 16GB, RTX 4080 16GB, RTX 4090 24GB) with zero layer-by-layer PCIe streaming bottlenecks.
* **Channel-Wise Level-0 Macro Scaling (`wcscales`)**:
  * Unlike standard scalar-scale quantization that suffers from SECAM-like chroma bleeding and dynamic range clipping, this checkpoint incorporates 224 channel-wise scale vectors ([4096] dimensions).
  * Boosts PSNR by **+4.62 dB** over scalar-scale quantization and raises Structural Similarity (SSIM) from 0.84 to 0.92+ across complex photorealistic scenes and high-frequency textures.
* **SVDQuant Rank-32 Error Compensation**:
  * Integrates a 16-bit low-rank residual branch alongside 4-bit normal float lattice quantization across all 32 transformer blocks (`attn.to_q`, `to_k`, `to_v`, `to_out.0`, `img_mlp.proj`, `img_mlp.gate_layer`, `img_mlp.out`).
* **Designed for Pairing with Heretic Text Encoder**:
  * Perfectly complements **[`Qwen21_Text_Encoder_Heretic`](https://huggingface.co/models/Qwen21_Text_Encoder_Heretic)** for maximum prompt adherence, eliminating vector deflection and maintaining fine semantic control.
* **Blazing Fast Local Inference**:
  * 25-step Text-to-Image (1024x1024) in **~20.0s** on a single RTX 5060 Ti (vs 64.8s on unquantized offloaded baselines — a **3.2x speedup**).
  * 40-step In-Context Image Editing (1024x1024) in **~41.0s** (vs 198.4s — a **4.8x speedup**).

---

## 📊 Benchmark & Quality Evaluation (RTX 5060 Ti 16GB)

### 1. Latency & Resource Utilization (1024 x 1024)

| Pipeline Stage | Baseline (BF16 Offloaded) | SVDQuant NVFP4 (Resident) | Speedup / Savings |
| :--- | :---: | :---: | :---: |
| **DiT Kernel Compute (25 steps)** | 23.26 s | **18.02 s** (0.72s / step) | **1.29x faster compute** |
| **PCIe Layer Offload Latency** | 37.40 s | **0.00 s** (100% Resident) | **Eradicated (37.4s saved!)** |
| **Total T2I Latency (25 steps)** | 64.84 s | **20.02 s** | **3.24x faster end-to-end** |
| **Total I2I Edit Latency (40 steps)** | 198.44 s | **41.02 s** | **4.84x faster end-to-end** |
| **Peak VRAM Footprint** | ~14.8 GB (offloaded) | **10.58 GB** (resident DiT + VAE + Streamer) | **4.92 GB Free Headroom on 16GB GPU** |

### 2. Reconstruction Fidelity (vs Unquantized BF16 Ground Truth)

| Domain | Baseline Scalar `wtscale` | Champion Channel-Wise `wcscales` | Fidelity Gain |
| :--- | :---: | :---: | :---: |
| **Animal Fur & Whiskers (Kitten T2I)** | 26.23 dB PSNR (0.9213 SSIM) | **28.87 dB PSNR (0.9402 SSIM)** | **+2.64 dB** (fuzziness eliminated) |
| **High-Res Facial Portraiture (Portrait T2I)** | 20.25 dB PSNR (0.8478 SSIM) | **24.87 dB PSNR (0.9193 SSIM)** | **+4.62 dB** (natural skin tones) |
| **In-Context Action Edit (Kitten Chase)** | 23.11 dB PSNR (0.8920 SSIM) | **25.93 dB PSNR (0.9510 SSIM)** | **+2.82 dB** (identity preserved) |
| **Broadcast Testcard (Chroma Spill Ratio)** | 4.58 (35x chroma bleed) | **0.88** (clean subcarrier edges) | **Banished SECAM-like bleed!** |

---

## 🚀 Quickstart Usage

### 1. Requirements & Setup

Ensure you have PyTorch, Diffusers, and Nunchaku installed:

```bash
pip install nunchaku diffusers transformers accelerate sentencepiece
```

### 2. Loading with Diffusers and Nunchaku

```python
import torch
from diffusers import QwenImage21Pipeline
from nunchaku.models.transformers.transformer_qwenimage21 import NunchakuQwenImage21Transformer2DModel

device = "cuda:0"

# 1. Load forged NVFP4 DiT directly into resident VRAM
transformer = NunchakuQwenImage21Transformer2DModel.from_pretrained(
    "svdq-fp4_r32-qwen-image-2.1.safetensors",
    device=device,
    torch_dtype=torch.bfloat16,
)

# 2. Assemble complete pipeline (pointing text encoder to Heretic if desired)
pipeline = QwenImage21Pipeline.from_pretrained(
    "Qwen/Qwen-Image-2.1",
    transformer=transformer,
    torch_dtype=torch.bfloat16,
)
pipeline.vae = pipeline.vae.to(device)
pipeline.vae.enable_tiling()

# 3. Generate high-resolution image
image = pipeline(
    prompt="A serene Japanese garden with blooming cherry blossoms, koi pond, and morning mist, highly detailed",
    height=1024,
    width=1024,
    num_inference_steps=25,
    true_cfg_scale=1.0,
).images[0]

image.save("serene_garden.png")
```

### 3. Pairing with Heretic Text Encoder (`Qwen21_Text_Encoder_Heretic`)

When generating scenes with complex human anatomy, athletic dynamics, or specific attire, pair this DiT with `Qwen21_Text_Encoder_Heretic` to eliminate VLM hesitation vector deflection and ensure exact prompt adherence:

```bash
# Launch high-throughput server with Heretic text encoder on port 4500:
./extras/imagegen_qwen21_nvfp4.sh 4500
```

---

## 📦 Packaged Sources & Tooling (`extras/`)

Inside the `extras/` folder of this repository, you will find:
* `ImageEditServer.py`: High-concurrency FastAPI/Uvicorn server supporting Text-to-Image (`/v1/images/generations`) and Multimodal In-Context Image Editing (`/v1/images/edits`).
* `QwenImage21NVFP4Backend.py`: Production backend with resident DiT management, VAE tiling, and optional PCIe layerwise text encoder streaming.
* `stream_encoder.py`: Zero-quality-loss layerwise weight streaming engine for Qwen3-VL-8B.
* `stream_quantize_qwen21.py`: Full quantization pipeline implementing alternating least-squares SVD rank-32 decomposition and channel-wise scale packing.
* `diagnose_trajectory_and_scales.py`: Step-by-step Euler flow-matching trajectory analyzer.
* `eval_divergence_metrics.py`: Multi-domain divergence evaluator (PSNR, SSIM, SigLIP cosine similarity, edge acuity, chroma spill).

---

## 📜 Citation & Credits

* **Qwen-Image-2.1**: Qwen Team, Alibaba Cloud.
* **SVDQuant & Nunchaku**: MIT HAN Lab (*SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models*).
* **Quantization & Channel-Scaling Optimization**: Oleg K. / Nikola Seeker Project.