Instructions to use madebyollin/texture-fix-vae-for-qwen-image-2.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use madebyollin/texture-fix-vae-for-qwen-image-2.1 with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("madebyollin/texture-fix-vae-for-qwen-image-2.1", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Rename to texture-fix-vae-for-qwen-image-2.1 (Qwen license §4(c)): rename ComfyUI file + demo images, update README/NOTICE
Browse files- .gitattributes +2 -0
- NOTICE +1 -1
- README.md +12 -12
- images/{decode-qwen-image-2.1-vae-texture-fix-full.png → decode-texture-fix-vae-full.png} +0 -0
- images/{decode-qwen-image-2.1-vae-texture-fix-zoomed-1.png → decode-texture-fix-vae-zoomed-1.png} +0 -0
- images/{decode-qwen-image-2.1-vae-texture-fix-zoomed-2.png → decode-texture-fix-vae-zoomed-2.png} +0 -0
- images/{decode-qwen-image-2.1-vae-texture-fix.png → decode-texture-fix-vae.png} +0 -0
- qwen_image_2.1_vae_texture_fix_bf16.safetensors → texture_fix_vae_for_qwen_image_2.1_bf16.safetensors +0 -0
.gitattributes
CHANGED
|
@@ -37,3 +37,5 @@ images/decode-qwen-image-2.1-vae-full.png filter=lfs diff=lfs merge=lfs -text
|
|
| 37 |
images/decode-qwen-image-2.1-vae-texture-fix-full.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
images/decode-qwen-image-2.1-vae-texture-fix.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
images/decode-qwen-image-2.1-vae.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 37 |
images/decode-qwen-image-2.1-vae-texture-fix-full.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
images/decode-qwen-image-2.1-vae-texture-fix.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
images/decode-qwen-image-2.1-vae.png filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
images/decode-texture-fix-vae-full.png filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
images/decode-texture-fix-vae.png filter=lfs diff=lfs merge=lfs -text
|
NOTICE
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd. All Rights Reserved.
|
| 2 |
|
| 3 |
-
Modified files: diffusion_pytorch_model.safetensors and
|
| 4 |
Qwen-Image-2.1 VAE (https://huggingface.co/Qwen/Qwen-Image-2.1) with modified (finetuned) decoder weights.
|
| 5 |
The modifications were made by madebyollin in 2026. The encoder weights are unchanged.
|
|
|
|
| 1 |
Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd. All Rights Reserved.
|
| 2 |
|
| 3 |
+
Modified files: diffusion_pytorch_model.safetensors and texture_fix_vae_for_qwen_image_2.1_bf16.safetensors contain the
|
| 4 |
Qwen-Image-2.1 VAE (https://huggingface.co/Qwen/Qwen-Image-2.1) with modified (finetuned) decoder weights.
|
| 5 |
The modifications were made by madebyollin in 2026. The encoder weights are unchanged.
|
README.md
CHANGED
|
@@ -9,11 +9,11 @@ tags:
|
|
| 9 |
- qwen-image
|
| 10 |
---
|
| 11 |
|
| 12 |
-
# Qwen-Image-2.1
|
| 13 |
|
| 14 |
-
|
| 15 |
|
| 16 |
-
|
| 17 |
|
| 18 |
<details>
|
| 19 |
<summary>Comparison Settings</summary>
|
|
@@ -24,17 +24,17 @@ The latents for the VAE comparison image below were generated by Qwen-Image-2.1
|
|
| 24 |
|
| 25 |
</details>
|
| 26 |
|
| 27 |
-
| Qwen-Image-2.1-VAE ([full-res](./images/decode-qwen-image-2.1-vae-full.png)) | 🪄
|
| 28 |
| --- | --- |
|
| 29 |
-
|  |  |  | .to("cuda")
|
| 47 |
```
|
| 48 |
|
| 49 |
# Mechanism
|
| 50 |
|
| 51 |
-
|
| 52 |
|
| 53 |
The TAESD recipe, like most image autoencoder training recipes, uses a mix of PSNR-focused (MSE/MAE), LPIPS, and adversarial (GAN) loss terms.
|
| 54 |
Whenever precise details can't be reconstructed, MSE/MAE loss encourages blurring, LPIPS loss encourages blurring+checkerboarding (among other artifacts), and adversarial loss encourages generating sharp/plausible (but fake) detail without obvious artifacts.
|
|
@@ -60,9 +60,9 @@ I suspect the original Qwen-Image-2.1-VAE was trained without a working adversar
|
|
| 60 |
|
| 61 |
# Metrics
|
| 62 |
|
| 63 |
-
|
| 64 |
|
| 65 |
-
| Metric | Qwen-Image-2.1-VAE |
|
| 66 |
| --- | --- | --- |
|
| 67 |
| rFID ↓ (COCO val2017, 5000 images @ 256²) | 3.37 | **2.08** |
|
| 68 |
| PSNR ↑ (COCO val2017 @ 256²) | **33.30** | 32.86 |
|
|
|
|
| 9 |
- qwen-image
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# Texture-Fix VAE for Qwen-Image-2.1
|
| 13 |
|
| 14 |
+
The Texture-Fix VAE is the [Qwen-Image-2.1 VAE](https://huggingface.co/Qwen/Qwen-Image-2.1/tree/main/vae), but finetuned to produce cleaner textures with no checkerboard artifacts.
|
| 15 |
|
| 16 |
+
The Texture-Fix VAE's improved decoding is most noticeable in detailed, photo-style images.
|
| 17 |
|
| 18 |
<details>
|
| 19 |
<summary>Comparison Settings</summary>
|
|
|
|
| 24 |
|
| 25 |
</details>
|
| 26 |
|
| 27 |
+
| Qwen-Image-2.1-VAE ([full-res](./images/decode-qwen-image-2.1-vae-full.png)) | 🪄 Texture-Fix VAE 🪄 ([full-res](./images/decode-texture-fix-vae-full.png)) |
|
| 28 |
| --- | --- |
|
| 29 |
+
|  |  |
|
| 30 |
+
|  |  |
|
| 31 |
+
|  |  |
|
| 32 |
|
| 33 |
# Usage
|
| 34 |
|
| 35 |
**ComfyUI**
|
| 36 |
|
| 37 |
+
Download [`texture_fix_vae_for_qwen_image_2.1_bf16.safetensors`](./texture_fix_vae_for_qwen_image_2.1_bf16.safetensors) into `ComfyUI/models/vae/` and select it in the `Load VAE` node, in place of `qwen_image_2.1_vae_bf16.safetensors`.
|
| 38 |
|
| 39 |
**🧨 Diffusers**
|
| 40 |
|
|
|
|
| 42 |
import torch
|
| 43 |
from diffusers import QwenImage21Pipeline, AutoencoderKLQwenImage21
|
| 44 |
|
| 45 |
+
vae = AutoencoderKLQwenImage21.from_pretrained("madebyollin/texture-fix-vae-for-qwen-image-2.1", torch_dtype=torch.bfloat16)
|
| 46 |
pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", vae=vae, torch_dtype=torch.bfloat16).to("cuda")
|
| 47 |
```
|
| 48 |
|
| 49 |
# Mechanism
|
| 50 |
|
| 51 |
+
The Texture-Fix VAE was created by finetuning the Qwen-Image-2.1 VAE decoder for ~5000 steps at learning rate 3e-5, with only the two highest-resolution decoder stages and output head unfrozen (7.5M trainable parameters), using the recipe developed for [TAESD](https://github.com/madebyollin/taesd).
|
| 52 |
|
| 53 |
The TAESD recipe, like most image autoencoder training recipes, uses a mix of PSNR-focused (MSE/MAE), LPIPS, and adversarial (GAN) loss terms.
|
| 54 |
Whenever precise details can't be reconstructed, MSE/MAE loss encourages blurring, LPIPS loss encourages blurring+checkerboarding (among other artifacts), and adversarial loss encourages generating sharp/plausible (but fake) detail without obvious artifacts.
|
|
|
|
| 60 |
|
| 61 |
# Metrics
|
| 62 |
|
| 63 |
+
The Texture-Fix VAE makes perceptual quality metrics (rFID) better and reconstruction accuracy metrics (LPIPS/PSNR) slightly worse.
|
| 64 |
|
| 65 |
+
| Metric | Qwen-Image-2.1-VAE | Texture-Fix VAE |
|
| 66 |
| --- | --- | --- |
|
| 67 |
| rFID ↓ (COCO val2017, 5000 images @ 256²) | 3.37 | **2.08** |
|
| 68 |
| PSNR ↑ (COCO val2017 @ 256²) | **33.30** | 32.86 |
|
images/{decode-qwen-image-2.1-vae-texture-fix-full.png → decode-texture-fix-vae-full.png}
RENAMED
|
File without changes
|
images/{decode-qwen-image-2.1-vae-texture-fix-zoomed-1.png → decode-texture-fix-vae-zoomed-1.png}
RENAMED
|
File without changes
|
images/{decode-qwen-image-2.1-vae-texture-fix-zoomed-2.png → decode-texture-fix-vae-zoomed-2.png}
RENAMED
|
File without changes
|
images/{decode-qwen-image-2.1-vae-texture-fix.png → decode-texture-fix-vae.png}
RENAMED
|
File without changes
|
qwen_image_2.1_vae_texture_fix_bf16.safetensors → texture_fix_vae_for_qwen_image_2.1_bf16.safetensors
RENAMED
|
File without changes
|