Drop measurement-rig detail from the no-baseline note

36e6f13 verified 28 days ago

2.93 kB

license: other
license_name: krea-2-community-license
license_link: >-
  https://cdn.jsdelivr.net/gh/krea-ai/krea-2@db3984fbc6e13b34c0064990fc2d95ac64d00058/assets/hf_samples/LICENSE.pdf
base_model: krea/Krea-2-Turbo
pipeline_tag: text-to-image
library_name: diffusers
tags:
  - krea
  - nunchaku
  - svdquant
  - int4
  - quantization

Krea 2 Turbo Nunchaku Lite INT4 r32

Diffusers-loadable INT4 conversion of krea/Krea-2-Turbo, quantized with diffuse-compressor. It loads with a plain from_pretrained call — no runtime graph patches and no extra runtime package.

The transformer uses quant_method: nunchaku_lite, INT4 SVDQ with group size 64, rank 32, 256 SVDQ targets and no AWQ W4A16 targets. Eight outer linears (img_in, time/text embedders, time_mod_proj, text_fusion.projector, final_layer.linear) stay in bf16, and the text_encoder component is BitsAndBytes 4-bit NF4 with bf16 compute. QKV projections are not fused, so this trades some speed for loading through the stock Diffusers graph. Calibrated on 32 prompts at 8 steps, 1024x1024.

Benchmark

Checkpoint	Latency	Max VRAM
This repo — Nunchaku Lite INT4 r32 + BNB4 text encoder	60.70 s (stdev 0.04 s)	19.97 GiB
Nunchaku Lite NVFP4 r32 + BNB4 text encoder	34.06 s (stdev 0.03 s)	20.34 GiB

NVIDIA RTX PRO 4000 Blackwell, settings as in the Run snippet below, one warmup and three measured runs, everything resident on the GPU with no offload. Latency covers the full pipeline call; VRAM is peak device usage.

There is no dense bf16 row because it does not fit on a 24 GiB card — the transformer alone is 26.3 GiB — while both quantized builds run resident. The NVFP4 build is 1.78x faster on this GPU, because NVIDIA RTX PRO 4000 Blackwell has native FP4 tensor cores that only NVFP4 can use. INT4 is the build for Turing through Ada, where NVFP4 does not run at all, and has not been benchmarked there.

Output Comparison

Dense reference (left) and this build (right). Same prompt, seed, scheduler, resolution, and step count. Pixel MAE 16.01 / RMSE 34.05, versus 8.18 / 19.92 for the NVFP4 build.

Run

Requires the Hugging Face kernels package and DIFFUSERS_TRUST_REMOTE_KERNELS=true, plus a Diffusers build with the nunchaku_lite quantizer. INT4 needs a Turing or newer NVIDIA GPU; Hopper is unsupported.

import torch
from diffusers import Krea2Pipeline

pipe = Krea2Pipeline.from_pretrained(
    "lite-infer/krea-2-turbo-nunchaku-lite-int4_r32-bnb4-text-encoder",
    torch_dtype=torch.bfloat16,
).to("cuda")

image = pipe(
    prompt="a fox in the snow",
    generator=torch.Generator("cuda").manual_seed(12345),
    width=1024,
    height=1024,
    num_inference_steps=8,
    guidance_scale=0.0,
).images[0]
image.save("output.png")