Instructions to use madebyollin/texture-fix-vae-for-qwen-image-2.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use madebyollin/texture-fix-vae-for-qwen-image-2.1 with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("madebyollin/texture-fix-vae-for-qwen-image-2.1", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Texture-Fix-VAE-for-Qwen-Image-2.1
Built with Qwen* (an unofficial finetune of the Qwen-Image-2.1 VAE)
Texture-Fix-VAE-for-Qwen-Image-2.1 is the Qwen-Image-2.1 VAE, but finetuned to produce cleaner textures with no checkerboard artifacts.
Comparison Settings
Texture-Fix-VAE-for-Qwen-Image-2.1's improved decoding is most noticeable in detailed, photo-style images. The latents for the VAE comparison image below were generated by Qwen-Image-2.1 from the photo-style prompt:
Landscape photograph of a subalpine wildflower meadow in the Pacific Northwest in midsummer: a clear mountain stream winding over mossy boulders through purple lupine and red paintbrush, dense old-growth Douglas fir and western red cedar forest behind, a snow-capped volcano in the distance, golden late-afternoon light, highly detailed
Usage
ComfyUI
Download texture_fix_vae_for_qwen_image_2.1_bf16.safetensors into ComfyUI/models/vae/ and select it in the Load VAE node, in place of qwen_image_2.1_vae_bf16.safetensors.
🧨 Diffusers
import torch
from diffusers import QwenImage21Pipeline, AutoencoderKLQwenImage21
vae = AutoencoderKLQwenImage21.from_pretrained("madebyollin/texture-fix-vae-for-qwen-image-2.1", torch_dtype=torch.bfloat16)
pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", vae=vae, torch_dtype=torch.bfloat16).to("cuda")
Mechanism
Texture-Fix-VAE-for-Qwen-Image-2.1 was created by finetuning the Qwen-Image-2.1 VAE decoder for ~5000 steps at learning rate 3e-5, with only the two highest-resolution decoder stages and output head unfrozen (7.5M trainable parameters), using the recipe developed for TAESD.
The TAESD recipe, like most image autoencoder training recipes, uses a mix of PSNR-focused (MSE/MAE), LPIPS, and adversarial (GAN) loss terms. Whenever precise details can't be reconstructed, MSE/MAE loss encourages blurring, LPIPS loss encourages blurring+checkerboarding (among other artifacts), and adversarial loss encourages generating sharp/plausible (but fake) detail without obvious artifacts. This figure from DC-AE shows the importance of including adversarial (GAN) loss:
Figure: from Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models (Chen et al., 2024, arXiv:2410.10733), licensed under CC BY 4.0; cropped to the first two rows.
I suspect the original Qwen-Image-2.1-VAE was trained without a working adversarial loss term.
Metrics
Texture-Fix-VAE-for-Qwen-Image-2.1 makes perceptual quality metrics (rFID) better and reconstruction accuracy metrics (LPIPS/PSNR) slightly worse.
| Metric | Qwen-Image-2.1-VAE | Texture-Fix-VAE-for-Qwen-Image-2.1 |
|---|---|---|
| rFID ↓ (COCO val2017, 5000 images @ 256²) | 3.37 | 2.08 |
| PSNR ↑ (COCO val2017 @ 256²) | 33.30 | 32.86 |
| LPIPS ↓ (COCO val2017 @ 256²) | 0.0357 | 0.0373 |
| PSNR ↑ (DIV2K valid, native 1024² crops) | 32.86 | 32.46 |
| LPIPS ↓ (DIV2K valid, native 1024² crops) | 0.0460 | 0.0480 |
Attribution Notice
This fine-tuned VAE is based on Qwen/Qwen-Image-2.1; original materials © 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd., licensed under the Qwen RESEARCH LICENSE AGREEMENT (see LICENSE) for non-commercial/research use only. Built with Qwen*.
* In the sense that the initial VAE weights are from Qwen-Image. The decoder fine-tuning work was performed by madebyollin and Claude Opus.
- Downloads last month
- 940






