|
3 | 3 | > **For more information about community pipelines, please have a look at [this issue](https://github.com/huggingface/diffusers/issues/841).** |
4 | 4 |
|
5 | 5 | **Community** examples consist of both inference and training examples that have been added by the community. |
| 6 | +Please have a look at the following table to get an overview of all community examples. Click on the **Code Example** to get a copy-and-paste ready code example that you can try out. |
| 7 | +If a community doesn't work as expected, please open an issue and ping the author on it. |
6 | 8 |
|
7 | | -| Example | Description | Author | Colab | |
8 | | -|:----------|:----------------------|:-----------------|----------:| |
9 | | -| CLIP Guided Stable Diffusion | Doing CLIP guidance for text to image generation with Stable Diffusion| [Suraj Patil](https://github.com/patil-suraj/) | [](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/CLIP_Guided_Stable_diffusion_with_diffusers.ipynb) | |
10 | | -| One Step U-Net (Dummy) | Example showcasing of how to use Community Pipelines (see https://github.com/huggingface/diffusers/issues/841) | [Patrick von Platen](https://github.com/patrickvonplaten/) | - | |
11 | | -| Stable Diffusion Interpolation | Interpolate the latent space of Stable Diffusion between different prompts/seeds | [Nate Raw](https://github.com/nateraw/) | - | |
| 9 | +| Example | Description | Code Example | Colab | Author | |
| 10 | +|:----------|:----------------------|:-----------------|:-------------|----------:| |
| 11 | +| CLIP Guided Stable Diffusion | Doing CLIP guidance for text to image generation with Stable Diffusion| [CLIP Guided Stable Diffusion](#clip-guided-stable-diffusion) | [](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/CLIP_Guided_Stable_diffusion_with_diffusers.ipynb) | [Suraj Patil](https://github.com/patil-suraj/) | |
| 12 | +| One Step U-Net (Dummy) | Example showcasing of how to use Community Pipelines (see https://github.com/huggingface/diffusers/issues/841) | [One Step U-Net](#one-step-unet) | - | [Patrick von Platen](https://github.com/patrickvonplaten/) | |
| 13 | +| Stable Diffusion Interpolation | Interpolate the latent space of Stable Diffusion between different prompts/seeds | [Stable Diffusion Interpolation](#stable-diffusion-interpolation) | [Nate Raw](https://github.com/nateraw/) | |
| 14 | +| Stable Diffusion Mega | **One** Stable Diffusion Pipeline with all functionalities of [Text2Image](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py), [Image2Image](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_img2img.py) and [Inpainting](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_inpaint.py) | [Stable Diffusion Mega](#stable-diffusion-mega) | - | [Patrick von Platen](https://github.com/patrickvonplaten/) | |
12 | 15 |
|
13 | 16 | ## Example usages |
14 | 17 |
|
@@ -66,7 +69,7 @@ Generated images tend to be of higher qualtiy than natively using stable diffusi |
66 | 69 |
|
67 | 70 | . |
68 | 71 |
|
69 | | -### One Step U-Net (Dummy) |
| 72 | +### One Step Unet |
70 | 73 |
|
71 | 74 | The dummy "one-step-unet" can be run as follows: |
72 | 75 |
|
@@ -112,3 +115,51 @@ frame_filepaths = pipe.walk( |
112 | 115 | The output of the `walk(...)` function returns a list of images saved under the folder as defined in `output_dir`. You can use these images to create videos of stable diffusion. |
113 | 116 |
|
114 | 117 | > **Please have a look at https://github.com/nateraw/stable-diffusion-videos for more in-detail information on how to create videos using stable diffusion as well as more feature-complete functionality.** |
| 118 | +
|
| 119 | +### Stable Diffusion Mega |
| 120 | + |
| 121 | +The Stable Diffusion Mega Pipeline lets you use the main use cases of the stable diffusion pipeline in a single class. |
| 122 | + |
| 123 | +```python |
| 124 | +#!/usr/bin/env python3 |
| 125 | +from diffusers import DiffusionPipeline |
| 126 | +import PIL |
| 127 | +import requests |
| 128 | +from io import BytesIO |
| 129 | +import torch |
| 130 | + |
| 131 | + |
| 132 | +def download_image(url): |
| 133 | + response = requests.get(url) |
| 134 | + return PIL.Image.open(BytesIO(response.content)).convert("RGB") |
| 135 | + |
| 136 | +pipe = DiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", custom_pipeline="stable_diffusion_mega", dtype=torch.float16, revision="fp16") |
| 137 | +pipe.to("cuda") |
| 138 | +pipe.enable_attention_slicing() |
| 139 | + |
| 140 | + |
| 141 | +### Text-to-Image |
| 142 | + |
| 143 | +images = pipe.text2img("An astronaut riding a horse").images |
| 144 | + |
| 145 | +### Image-to-Image |
| 146 | + |
| 147 | +init_image = download_image("https://raw.githubusercontent.com/CompVis/stable-diffusion/main/assets/stable-samples/img2img/sketch-mountains-input.jpg") |
| 148 | + |
| 149 | +prompt = "A fantasy landscape, trending on artstation" |
| 150 | + |
| 151 | +images = pipe.img2img(prompt=prompt, init_image=init_image, strength=0.75, guidance_scale=7.5).images |
| 152 | + |
| 153 | +### Inpainting |
| 154 | + |
| 155 | +img_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png" |
| 156 | +mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png" |
| 157 | +init_image = download_image(img_url).resize((512, 512)) |
| 158 | +mask_image = download_image(mask_url).resize((512, 512)) |
| 159 | + |
| 160 | +prompt = "a cat sitting on a bench" |
| 161 | +images = pipe.inpaint(prompt=prompt, init_image=init_image, mask_image=mask_image, strength=0.75).images |
| 162 | +``` |
| 163 | + |
| 164 | +As shown above this one pipeline can run all both "text-to-image", "image-to-image", and "inpainting" in one pipeline. |
| 165 | + |
0 commit comments