File size: 1,516 Bytes
0f7a765 e8263c1 bd1678a 0f7a765 b9f05dc 0f7a765 b9f05dc | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | ---
license: apache-2.0
base_model:
- vladmandic/Qwen-Lightning
base_model_relation: quantized
library_name: diffusers
tags:
- sdnq
- qwen_image
- 4-bit
---
4 bit (UINT4 with SVD rank 32) quantization of [vladmandic/Qwen-Lightning](https://huggingface.co/vladmandic/Qwen-Lightning) using [SDNQ](https://github.com/vladmandic/sdnext/wiki/SDNQ-Quantization).
Usage:
```
pip install git+https://github.com/Disty0/sdnq
```
```py
import torch
import diffusers
from sdnq import SDNQConfig # import sdnq to register it into diffusers and transformers
pipe = diffusers.QwenImagePipeline.from_pretrained("Disty0/Qwen-Image-Lightning-SDNQ-uint4-svd-r32", torch_dtype=torch.bfloat16)
pipe.enable_model_cpu_offload()
prompt = "a tiny astronaut hatching from an egg on the moon, Ultra HD, 4K, cinematic composition."
negative_prompt = " "
image = pipe(
prompt=prompt,
negative_prompt=negative_prompt,
width=1024,
height=1024,
num_inference_steps=8,
true_cfg_scale=1.0,
generator=torch.manual_seed(0),
).images[0]
image.save("qwen-image-lightning-sdnq-uint4-svd-r32.png")
```
Original BF16 vs SDNQ quantization comparison:
| Quantization | Model Size | Visualization |
| --- | --- | --- |
| Original BF16 | 40.9 GB |  |
| SDNQ UINT4 | 11.6 GB |  | |