This is awesome!
I have been following this LoRA since the first time you posted about it, but this final version is really awesome. Tried to combine with my style LoRAs and I didn't notice any degradation or drift from the style. Thank you for your work!
PS: You might want to consider also uploading the entire DiT with the LoRA pre-merged and then quantized to INT8-ConvRot. This would have slightly better precision than applying the LoRA to quantized weights since, afaik ComfyUI dequantizes, applies and requantizes.
All my tests are with fused DiT, I just assume everyone would know how to do that plus I don't want to keep 'copies' of Krea models under my repo, licensing covers the LoRA part but don't want to do full DiT. Otherwise agree with you, pre merged is the way to go. And glad you like it!
All my tests are with fused DiT, I just assume everyone would know how to do that plus I don't want to keep 'copies' of Krea models under my repo, licensing covers the LoRA part but don't want to do full DiT. Otherwise agree with you, pre merged is the way to go. And glad you like it!
How would you recommend pre-merging this LoRA with a Krea 2 model?
All my tests are with fused DiT, I just assume everyone would know how to do that plus I don't want to keep 'copies' of Krea models under my repo, licensing covers the LoRA part but don't want to do full DiT. Otherwise agree with you, pre merged is the way to go. And glad you like it!
How would you recommend pre-merging this LoRA with a Krea 2 model?
Fusing a LoRA into a DiT is just adding the low-rank update back into each target weight.
Do it in bf16 or fp32 on the unquantized base, then quantize the result. That is the whole point of pre-merging: quantizing after the fusion keeps the LoRA's contribution inside the quantizer's error budget, instead of ComfyUI dequantizing, applying and requantizing at load time.
Practical routes:
- ComfyUI, no code: load the base model, LoraLoaderModelOnly at strength 1.0, then Save Checkpoint. Quantize that file afterwards.
- diffusers: pipe.load_lora_weights(...) then pipe.fuse_lora(), and pipe.transformer.save_pretrained(...).