Skip to content

Commit 36db6a8

Browse files
Merge branch 'main' into unet_argument_checking
2 parents 03740de + a688c7b commit 36db6a8

File tree

2 files changed

+611
-0
lines changed

2 files changed

+611
-0
lines changed

examples/community/README.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,8 @@ If a community doesn't work as expected, please open an issue and ping the autho
2727
Stable Diffusion v1.1-1.4 Comparison | Run all 4 model checkpoints for Stable Diffusion and compare their results together | [Stable Diffusion Comparison](#stable-diffusion-comparisons) | - | [Suvaditya Mukherjee](https://github.com/suvadityamuk) |
2828
MagicMix | Diffusion Pipeline for semantic mixing of an image and a text prompt | [MagicMix](#magic-mix) | - | [Partho Das](https://github.com/daspartho) |
2929
| Stable UnCLIP | Diffusion Pipeline for combining prior model (generate clip image embedding from text, UnCLIPPipeline `"kakaobrain/karlo-v1-alpha"`) and decoder pipeline (decode clip image embedding to image, StableDiffusionImageVariationPipeline `"lambdalabs/sd-image-variations-diffusers"` ). | [Stable UnCLIP](#stable-unclip) | - |[Ray Wang](https://wrong.wang) |
30+
| UnCLIP Text Interpolation Pipeline | Diffusion Pipeline that allows passing two prompts and produces images while interpolating between the text-embeddings of the two prompts | [UnCLIP Text Interpolation Pipeline](#unclip-text-interpolation-pipeline) | - | [Naga Sai Abhinay Devarinti](https://github.com/Abhinay1997/) |
31+
3032

3133

3234

@@ -951,3 +953,39 @@ print(pipeline.prior_scheduler)
951953

952954
![shiba-inu](https://user-images.githubusercontent.com/16448529/209185639-6e5ec794-ce9d-4883-aa29-bd6852a2abad.jpg)
953955

956+
### UnCLIP Text Interpolation Pipeline
957+
958+
This Diffusion Pipeline takes two prompts and interpolates between the two input prompts using spherical interpolation ( slerp ). The input prompts are converted to text embeddings by the pipeline's text_encoder and the interpolation is done on the resulting text_embeddings over the number of steps specified. Defaults to 5 steps.
959+
960+
```python
961+
import torch
962+
from diffusers import DiffusionPipeline
963+
964+
device = torch.device("cpu" if not torch.cuda.is_available() else "cuda")
965+
966+
pipe = DiffusionPipeline.from_pretrained(
967+
"kakaobrain/karlo-v1-alpha",
968+
torch_dtype=torch.float16,
969+
custom_pipeline="unclip_text_interpolation"
970+
)
971+
pipe.to(device)
972+
973+
start_prompt = "A photograph of an adult lion"
974+
end_prompt = "A photograph of a lion cub"
975+
#For best results keep the prompts close in length to each other. Of course, feel free to try out with differing lengths.
976+
generator = torch.Generator(device=device).manual_seed(42)
977+
978+
output = pipe(start_prompt, end_prompt, steps = 6, generator = generator, enable_sequential_cpu_offload=False)
979+
980+
for i,image in enumerate(output.images):
981+
img.save('result%s.jpg' % i)
982+
```
983+
984+
The resulting images in order:-
985+
986+
![result_0](https://huggingface.co/datasets/NagaSaiAbhinay/UnCLIPTextInterpolationSamples/resolve/main/lion_to_cub_0.png)
987+
![result_1](https://huggingface.co/datasets/NagaSaiAbhinay/UnCLIPTextInterpolationSamples/resolve/main/lion_to_cub_1.png)
988+
![result_2](https://huggingface.co/datasets/NagaSaiAbhinay/UnCLIPTextInterpolationSamples/resolve/main/lion_to_cub_2.png)
989+
![result_3](https://huggingface.co/datasets/NagaSaiAbhinay/UnCLIPTextInterpolationSamples/resolve/main/lion_to_cub_3.png)
990+
![result_4](https://huggingface.co/datasets/NagaSaiAbhinay/UnCLIPTextInterpolationSamples/resolve/main/lion_to_cub_4.png)
991+
![result_5](https://huggingface.co/datasets/NagaSaiAbhinay/UnCLIPTextInterpolationSamples/resolve/main/lion_to_cub_5.png)

0 commit comments

Comments
 (0)