Diffusers
Safetensors
vae
qwen-image
madebyollin commited on
Commit
723410f
·
verified ·
1 Parent(s): 5276abd

Rename to texture-fix-vae-for-qwen-image-2.1 (Qwen license §4(c)): rename ComfyUI file + demo images, update README/NOTICE

Browse files
.gitattributes CHANGED
@@ -37,3 +37,5 @@ images/decode-qwen-image-2.1-vae-full.png filter=lfs diff=lfs merge=lfs -text
37
  images/decode-qwen-image-2.1-vae-texture-fix-full.png filter=lfs diff=lfs merge=lfs -text
38
  images/decode-qwen-image-2.1-vae-texture-fix.png filter=lfs diff=lfs merge=lfs -text
39
  images/decode-qwen-image-2.1-vae.png filter=lfs diff=lfs merge=lfs -text
 
 
 
37
  images/decode-qwen-image-2.1-vae-texture-fix-full.png filter=lfs diff=lfs merge=lfs -text
38
  images/decode-qwen-image-2.1-vae-texture-fix.png filter=lfs diff=lfs merge=lfs -text
39
  images/decode-qwen-image-2.1-vae.png filter=lfs diff=lfs merge=lfs -text
40
+ images/decode-texture-fix-vae-full.png filter=lfs diff=lfs merge=lfs -text
41
+ images/decode-texture-fix-vae.png filter=lfs diff=lfs merge=lfs -text
NOTICE CHANGED
@@ -1,5 +1,5 @@
1
  Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd. All Rights Reserved.
2
 
3
- Modified files: diffusion_pytorch_model.safetensors and qwen_image_2.1_vae_texture_fix_bf16.safetensors contain the
4
  Qwen-Image-2.1 VAE (https://huggingface.co/Qwen/Qwen-Image-2.1) with modified (finetuned) decoder weights.
5
  The modifications were made by madebyollin in 2026. The encoder weights are unchanged.
 
1
  Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd. All Rights Reserved.
2
 
3
+ Modified files: diffusion_pytorch_model.safetensors and texture_fix_vae_for_qwen_image_2.1_bf16.safetensors contain the
4
  Qwen-Image-2.1 VAE (https://huggingface.co/Qwen/Qwen-Image-2.1) with modified (finetuned) decoder weights.
5
  The modifications were made by madebyollin in 2026. The encoder weights are unchanged.
README.md CHANGED
@@ -9,11 +9,11 @@ tags:
9
  - qwen-image
10
  ---
11
 
12
- # Qwen-Image-2.1-VAE-Texture-Fix
13
 
14
- Qwen-Image-2.1-VAE-Texture-Fix is the [Qwen-Image-2.1 VAE](https://huggingface.co/Qwen/Qwen-Image-2.1/tree/main/vae), but finetuned to produce cleaner textures with no checkerboard artifacts.
15
 
16
- Qwen-Image-2.1-VAE-Texture-Fix's improved decoding is most noticeable in detailed, photo-style images.
17
 
18
  <details>
19
  <summary>Comparison Settings</summary>
@@ -24,17 +24,17 @@ The latents for the VAE comparison image below were generated by Qwen-Image-2.1
24
 
25
  </details>
26
 
27
- | Qwen-Image-2.1-VAE ([full-res](./images/decode-qwen-image-2.1-vae-full.png)) | 🪄 Qwen-Image-2.1-VAE-Texture-Fix 🪄 ([full-res](./images/decode-qwen-image-2.1-vae-texture-fix-full.png)) |
28
  | --- | --- |
29
- | ![](./images/decode-qwen-image-2.1-vae-zoomed-1.png) | ![](./images/decode-qwen-image-2.1-vae-texture-fix-zoomed-1.png) |
30
- | ![](./images/decode-qwen-image-2.1-vae-zoomed-2.png) | ![](./images/decode-qwen-image-2.1-vae-texture-fix-zoomed-2.png) |
31
- | ![](./images/decode-qwen-image-2.1-vae.png) | ![](./images/decode-qwen-image-2.1-vae-texture-fix.png) |
32
 
33
  # Usage
34
 
35
  **ComfyUI**
36
 
37
- Download [`qwen_image_2.1_vae_texture_fix_bf16.safetensors`](./qwen_image_2.1_vae_texture_fix_bf16.safetensors) into `ComfyUI/models/vae/` and select it in the `Load VAE` node, in place of `qwen_image_2.1_vae_bf16.safetensors`.
38
 
39
  **🧨 Diffusers**
40
 
@@ -42,13 +42,13 @@ Download [`qwen_image_2.1_vae_texture_fix_bf16.safetensors`](./qwen_image_2.1_va
42
  import torch
43
  from diffusers import QwenImage21Pipeline, AutoencoderKLQwenImage21
44
 
45
- vae = AutoencoderKLQwenImage21.from_pretrained("madebyollin/qwen-image-2.1-vae-texture-fix", torch_dtype=torch.bfloat16)
46
  pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", vae=vae, torch_dtype=torch.bfloat16).to("cuda")
47
  ```
48
 
49
  # Mechanism
50
 
51
- Qwen-Image-2.1-VAE-Texture-Fix was created by finetuning the Qwen-Image-2.1 VAE decoder for ~5000 steps at learning rate 3e-5, with only the two highest-resolution decoder stages and output head unfrozen (7.5M trainable parameters), using the recipe developed for [TAESD](https://github.com/madebyollin/taesd).
52
 
53
  The TAESD recipe, like most image autoencoder training recipes, uses a mix of PSNR-focused (MSE/MAE), LPIPS, and adversarial (GAN) loss terms.
54
  Whenever precise details can't be reconstructed, MSE/MAE loss encourages blurring, LPIPS loss encourages blurring+checkerboarding (among other artifacts), and adversarial loss encourages generating sharp/plausible (but fake) detail without obvious artifacts.
@@ -60,9 +60,9 @@ I suspect the original Qwen-Image-2.1-VAE was trained without a working adversar
60
 
61
  # Metrics
62
 
63
- Qwen-Image-2.1-VAE-Texture-Fix makes perceptual quality metrics (rFID) better and reconstruction accuracy metrics (LPIPS/PSNR) slightly worse.
64
 
65
- | Metric | Qwen-Image-2.1-VAE | Qwen-Image-2.1-VAE-Texture-Fix |
66
  | --- | --- | --- |
67
  | rFID ↓ (COCO val2017, 5000 images @ 256²) | 3.37 | **2.08** |
68
  | PSNR ↑ (COCO val2017 @ 256²) | **33.30** | 32.86 |
 
9
  - qwen-image
10
  ---
11
 
12
+ # Texture-Fix VAE for Qwen-Image-2.1
13
 
14
+ The Texture-Fix VAE is the [Qwen-Image-2.1 VAE](https://huggingface.co/Qwen/Qwen-Image-2.1/tree/main/vae), but finetuned to produce cleaner textures with no checkerboard artifacts.
15
 
16
+ The Texture-Fix VAE's improved decoding is most noticeable in detailed, photo-style images.
17
 
18
  <details>
19
  <summary>Comparison Settings</summary>
 
24
 
25
  </details>
26
 
27
+ | Qwen-Image-2.1-VAE ([full-res](./images/decode-qwen-image-2.1-vae-full.png)) | 🪄 Texture-Fix VAE 🪄 ([full-res](./images/decode-texture-fix-vae-full.png)) |
28
  | --- | --- |
29
+ | ![](./images/decode-qwen-image-2.1-vae-zoomed-1.png) | ![](./images/decode-texture-fix-vae-zoomed-1.png) |
30
+ | ![](./images/decode-qwen-image-2.1-vae-zoomed-2.png) | ![](./images/decode-texture-fix-vae-zoomed-2.png) |
31
+ | ![](./images/decode-qwen-image-2.1-vae.png) | ![](./images/decode-texture-fix-vae.png) |
32
 
33
  # Usage
34
 
35
  **ComfyUI**
36
 
37
+ Download [`texture_fix_vae_for_qwen_image_2.1_bf16.safetensors`](./texture_fix_vae_for_qwen_image_2.1_bf16.safetensors) into `ComfyUI/models/vae/` and select it in the `Load VAE` node, in place of `qwen_image_2.1_vae_bf16.safetensors`.
38
 
39
  **🧨 Diffusers**
40
 
 
42
  import torch
43
  from diffusers import QwenImage21Pipeline, AutoencoderKLQwenImage21
44
 
45
+ vae = AutoencoderKLQwenImage21.from_pretrained("madebyollin/texture-fix-vae-for-qwen-image-2.1", torch_dtype=torch.bfloat16)
46
  pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", vae=vae, torch_dtype=torch.bfloat16).to("cuda")
47
  ```
48
 
49
  # Mechanism
50
 
51
+ The Texture-Fix VAE was created by finetuning the Qwen-Image-2.1 VAE decoder for ~5000 steps at learning rate 3e-5, with only the two highest-resolution decoder stages and output head unfrozen (7.5M trainable parameters), using the recipe developed for [TAESD](https://github.com/madebyollin/taesd).
52
 
53
  The TAESD recipe, like most image autoencoder training recipes, uses a mix of PSNR-focused (MSE/MAE), LPIPS, and adversarial (GAN) loss terms.
54
  Whenever precise details can't be reconstructed, MSE/MAE loss encourages blurring, LPIPS loss encourages blurring+checkerboarding (among other artifacts), and adversarial loss encourages generating sharp/plausible (but fake) detail without obvious artifacts.
 
60
 
61
  # Metrics
62
 
63
+ The Texture-Fix VAE makes perceptual quality metrics (rFID) better and reconstruction accuracy metrics (LPIPS/PSNR) slightly worse.
64
 
65
+ | Metric | Qwen-Image-2.1-VAE | Texture-Fix VAE |
66
  | --- | --- | --- |
67
  | rFID ↓ (COCO val2017, 5000 images @ 256²) | 3.37 | **2.08** |
68
  | PSNR ↑ (COCO val2017 @ 256²) | **33.30** | 32.86 |
images/{decode-qwen-image-2.1-vae-texture-fix-full.png → decode-texture-fix-vae-full.png} RENAMED
File without changes
images/{decode-qwen-image-2.1-vae-texture-fix-zoomed-1.png → decode-texture-fix-vae-zoomed-1.png} RENAMED
File without changes
images/{decode-qwen-image-2.1-vae-texture-fix-zoomed-2.png → decode-texture-fix-vae-zoomed-2.png} RENAMED
File without changes
images/{decode-qwen-image-2.1-vae-texture-fix.png → decode-texture-fix-vae.png} RENAMED
File without changes
qwen_image_2.1_vae_texture_fix_bf16.safetensors → texture_fix_vae_for_qwen_image_2.1_bf16.safetensors RENAMED
File without changes