Skip to content

Commit 523a50a

Browse files
authored
[docs] Load A1111 LoRA (#3629)
* load a1111 lora * fix * apply feedback * fix
1 parent de45af4 commit 523a50a

File tree

1 file changed

+66
-1
lines changed

1 file changed

+66
-1
lines changed

docs/source/en/using-diffusers/other-formats.mdx

Lines changed: 66 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,4 +123,69 @@ pipeline.to("cuda")
123123
placeholder_token = "<my-funny-cat-token>"
124124
prompt = f"two {placeholder_token} getting married, photorealistic, high quality"
125125
image = pipeline(prompt, num_inference_steps=50).images[0]
126-
```
126+
```
127+
128+
## A1111 LoRA files
129+
130+
[Automatic1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui) (A1111) is a popular web UI for Stable Diffusion that supports model sharing platforms like [Civitai](https://civitai.com/). Models trained with the Low-Rank Adaptation (LoRA) technique are especially popular because they're fast to train and have a much smaller file size than a fully finetuned model. 🤗 Diffusers supports loading A1111 LoRA checkpoints with [`~LoraLoaderMixin.load_lora_weights`]:
131+
132+
```py
133+
from diffusers import DiffusionPipeline, UniPCMultistepScheduler
134+
import torch
135+
136+
pipeline = DiffusionPipeline.from_pretrained(
137+
"andite/anything-v4.0", torch_dtype=torch.float16, safety_checker=None
138+
).to("cuda")
139+
pipeline.scheduler = UniPCMultistepScheduler.from_config(pipeline.scheduler.config)
140+
```
141+
142+
Download a LoRA checkpoint from Civitai; this example uses the [Howls Moving Castle,Interior/Scenery LoRA (Ghibli Stlye)](https://civitai.com/models/14605?modelVersionId=19998) checkpoint, but feel free to try out any LoRA checkpoint!
143+
144+
```bash
145+
!wget https://civitai.com/api/download/models/19998 -O howls_moving_castle.safetensors
146+
```
147+
148+
Load the LoRA checkpoint into the pipeline with the [`~LoraLoaderMixin.load_lora_weights`] method:
149+
150+
```py
151+
pipeline.load_lora_weights(".", weight_name="howls_moving_castle.safetensors")
152+
```
153+
154+
Now you can use the pipeline to generate images:
155+
156+
```py
157+
prompt = "masterpiece, illustration, ultra-detailed, cityscape, san francisco, golden gate bridge, california, bay area, in the snow, beautiful detailed starry sky"
158+
negative_prompt = "lowres, cropped, worst quality, low quality, normal quality, artifacts, signature, watermark, username, blurry, more than one bridge, bad architecture"
159+
160+
images = pipeline(
161+
prompt=prompt,
162+
negative_prompt=negative_prompt,
163+
width=512,
164+
height=512,
165+
num_inference_steps=25,
166+
num_images_per_prompt=4,
167+
generator=torch.manual_seed(0),
168+
).images
169+
```
170+
171+
Finally, create a helper function to display the images:
172+
173+
```py
174+
from PIL import Image
175+
176+
177+
def image_grid(imgs, rows=2, cols=2):
178+
w, h = imgs[0].size
179+
grid = Image.new("RGB", size=(cols * w, rows * h))
180+
181+
for i, img in enumerate(imgs):
182+
grid.paste(img, box=(i % cols * w, i // cols * h))
183+
return grid
184+
185+
186+
image_grid(images)
187+
```
188+
189+
<div class="flex justify-center">
190+
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/a1111-lora-sf.png"/>
191+
</div>

0 commit comments

Comments
 (0)