File size: 2,928 Bytes
f1c16af 129b6c5 f1c16af 36e6f13 f1c16af 129b6c5 f1c16af | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | ---
license: other
license_name: krea-2-community-license
license_link: https://cdn.jsdelivr.net/gh/krea-ai/krea-2@db3984fbc6e13b34c0064990fc2d95ac64d00058/assets/hf_samples/LICENSE.pdf
base_model: krea/Krea-2-Turbo
pipeline_tag: text-to-image
library_name: diffusers
tags:
- krea
- nunchaku
- svdquant
- int4
- quantization
---
# Krea 2 Turbo Nunchaku Lite INT4 r32
Diffusers-loadable INT4 conversion of `krea/Krea-2-Turbo`, quantized with
[`diffuse-compressor`](https://github.com/rootonchair/diffuse-compressor). It
loads with a plain `from_pretrained` call — no runtime graph patches and no extra
runtime package.
The transformer uses `quant_method: nunchaku_lite`, INT4 SVDQ with group size
64, rank 32, 256 SVDQ targets and no AWQ W4A16 targets. Eight outer linears (`img_in`, time/text embedders, `time_mod_proj`, `text_fusion.projector`, `final_layer.linear`) stay
in bf16, and the `text_encoder` component is BitsAndBytes 4-bit NF4 with bf16
compute. QKV projections are not fused, so this trades some speed for loading
through the stock Diffusers graph. Calibrated on 32 prompts at
8 steps, 1024x1024.
## Benchmark
| Checkpoint | Latency | Max VRAM |
| --- | ---: | ---: |
| **This repo** — Nunchaku Lite INT4 r32 + BNB4 text encoder | 60.70 s (stdev 0.04 s) | 19.97 GiB |
| Nunchaku Lite NVFP4 r32 + BNB4 text encoder | 34.06 s (stdev 0.03 s) | 20.34 GiB |
NVIDIA RTX PRO 4000 Blackwell, settings as in the Run snippet below, one warmup and three
measured runs, everything resident on the GPU with no offload. Latency covers
the full pipeline call; VRAM is peak device usage.
There is no dense bf16 row because it does not fit on a 24 GiB card — the transformer alone is 26.3 GiB — while both quantized builds run resident. The NVFP4 build is 1.78x faster **on this GPU**, because NVIDIA RTX PRO 4000 Blackwell has native FP4 tensor cores that only NVFP4 can use. **INT4 is the build for Turing through Ada, where NVFP4 does not run at all**, and has not been benchmarked there.
## Output Comparison
Dense reference (left) and this build (right). Same prompt, seed, scheduler, resolution, and step count. Pixel **MAE 16.01 / RMSE 34.05**, versus 8.18 / 19.92 for the NVFP4 build.
## Run
Requires the Hugging Face `kernels` package and `DIFFUSERS_TRUST_REMOTE_KERNELS=true`,
plus a Diffusers build with the `nunchaku_lite` quantizer. INT4 needs
a Turing or newer NVIDIA GPU; Hopper is unsupported.
```python
import torch
from diffusers import Krea2Pipeline
pipe = Krea2Pipeline.from_pretrained(
"lite-infer/krea-2-turbo-nunchaku-lite-int4_r32-bnb4-text-encoder",
torch_dtype=torch.bfloat16,
).to("cuda")
image = pipe(
prompt="a fox in the snow",
generator=torch.Generator("cuda").manual_seed(12345),
width=1024,
height=1024,
num_inference_steps=8,
guidance_scale=0.0,
).images[0]
image.save("output.png")
```
|