File size: 2,552 Bytes
ac9337d 13f7591 ac9337d 13f7591 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | ---
language:
- en
pipeline_tag: unconditional-image-generation
tags:
- Diffusion Models
- Stable Diffusion
- Perturbed-Attention Guidance
- PAG
---
# Perturbed-Attention Guidance for SDXL
<div style="display:flex">
<video width=50% autoplay loop controls>
<source src="https://huggingface.co/multimodalart/sdxl_perturbed_attention_guidance/resolve/main/pag_sdxl.mp4" type="video/mp4">
</video>
<video width=50% autoplay loop controls>
<source src="https://huggingface.co/multimodalart/sdxl_perturbed_attention_guidance/resolve/main/pag_uncond.mp4" type="video/mp4">
</video>
</div>
The original Perturbed-Attention Guidance for unconditional models and SD1.5 by [Hyoungwon Cho](https://huggingface.co/hyoungwoncho) is availiable at [hyoungwoncho/sd_perturbed_attention_guidance](https://huggingface.co/hyoungwoncho/sd_perturbed_attention_guidance)
[Project](https://ku-cvlab.github.io/Perturbed-Attention-Guidance/) / [arXiv](https://arxiv.org/abs/2403.17377) / [GitHub](https://github.com/KU-CVLAB/Perturbed-Attention-Guidance)
This repository is just a simple SDXL implementation of the Perturbed-Attention Guidance (PAG) on Stable Diffusion XL (SDXL) for the 🧨 diffusers library.
## Quickstart
Loading Custom Pipeline:
```py
from diffusers import StableDiffusionXLPipeline
pipe = StableDiffusionXLPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
custom_pipeline="multimodalart/sdxl_perturbed_attention_guidance",
torch_dtype=torch.float16
)
device="cuda"
pipe = pipe.to(device)
```
Unconditional sampling with PAG:
```py
output = pipe(
"",
num_inference_steps=50,
guidance_scale=0.0,
pag_scale=5.0,
pag_applied_layers=['mid']
).images
```
Sampling with PAG and CFG:
```py
output = pipe(
"the spirit of a tamagotchi wandering in the city of Vienna",
num_inference_steps=25,
guidance_scale=4.0,
pag_scale=3.0,
pag_applied_layers=['mid']
).images
```
## Parameters
`guidance_scale` : guidance scale of CFG (ex: `7.5`)
`pag_scale` : guidance scale of PAG (ex: `4.0`)
`pag_applied_layers`: layer to apply perturbation (ex: ['mid'])
`pag_applied_layers_index` : index of the layers to apply perturbation (ex: ['m0', 'm1'])
## Stable Diffusion XL Demo
[Try it here](https://huggingface.co/spaces/multimodalart/perturbed-attention-guidance-sdxl) |