metadata
license: openrail++
base_model:
- stabilityai/stable-diffusion-xl-base-1.0
base_model_relation: finetune
pipeline_tag: text-to-image
library_name: diffusers
tags:
- stable-diffusion-xl
- preference-optimization
- pickscore
- diffusion-lair
Model details
- Base model:
stabilityai/stable-diffusion-xl-base-1.0 - Base-model revision:
462165984030d82259a11f4367a4eed129e94a7b - Architecture: Stable Diffusion XL
- Resolution: 1024 × 1024
- Fine-tuned component: UNet
- Weight format: sharded Safetensors
- Weight precision: float32
- Training objective: Diffusion LAIR
- Reward/preference model: PickScore
- Training dataset: Pick-a-Pic v2
License
This model is distributed under the CreativeML Open RAIL++-M license. Users must comply with the use-based restrictions specified in the license.
Usage
import torch
from diffusers import (
AutoencoderKL,
StableDiffusionXLPipeline,
UNet2DConditionModel,
)
model_id = "austin-k-wang/DiffusionLAIR-SDXL"
base_model_id = "stabilityai/stable-diffusion-xl-base-1.0"
unet = UNet2DConditionModel.from_pretrained(
model_id,
subfolder="unet",
torch_dtype=torch.float16,
use_safetensors=True,
)
vae = AutoencoderKL.from_pretrained(
"madebyollin/sdxl-vae-fp16-fix",
torch_dtype=torch.float16,
)
pipe = StableDiffusionXLPipeline.from_pretrained(
base_model_id,
unet=unet,
vae=vae,
variant="fp16",
torch_dtype=torch.float16,
use_safetensors=True,
).to("cuda")
generator = torch.Generator(device="cuda").manual_seed(42)
image = pipe(
"a cinematic photograph of a futuristic city at sunset",
generator=generator,
guidance_scale=5.0,
num_inference_steps=30,
).images[0]
image.save("output.png")
For lower-VRAM inference, replace .to("cuda") with:
pipe.enable_model_cpu_offload()