-
Notifications
You must be signed in to change notification settings - Fork 6.6k
Closed
Labels
Good second issuecontributions-welcomegood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is needed
Description
We recently added IP-adapter support to many of our pipelines in diffusers! You can now very easily load your IP-Adapter into a diffusers pipeline with pipe.load_ip_adapter(...);
However right now we do not support IPAdapterFull model checkpoints. Anyone interested to add that support for this? It will allow us to use the very cool face models with diffusers!
the goal would be to make this example work with diffusers API. e.g.
from diffusers import StableDiffusionPipeline
import torch
from diffusers.utils import load_image
base_model_path = "dreamlike-art/dreamlike-anime-1.0/"
pipeline = StableDiffusionPipeline.from_pretrained(
base_model_path, torch_dtype=torch.float16)
pipeline.to("cuda")
image = load_image("https://huggingface.co/datasets/YiYiXu/testing-images/resolve/main/ai_face2.png")
pipeline.load_ip_adapter("h94/IP-Adapter", subfolder="models", weight_name="ip-adapter-full-face_sd15.bin")
pipeline.set_ip_adapter_scale(0.6)
generator = torch.Generator(device="cpu").manual_seed(33)
images = pipeline(
prompt="A photo of a girl wearing a black dress, holding red roses in hand, upper body, behind is the Eiffel Tower",
ip_adapter_image=image,
negative_prompt="monochrome, lowres, bad anatomy, worst quality, low quality",
num_inference_steps=50,
generator=generator,
).images
images[0]You will need to look into this PR to understand how load_ip_adapter works and then make it work for IPAdapterFull model from the original repo.
jroessler
Metadata
Metadata
Assignees
Labels
Good second issuecontributions-welcomegood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is needed