-
Notifications
You must be signed in to change notification settings - Fork 6.1k
Closed
Description
LoRA loading is supported in the StableDiffusionPipeline
thanks to the last release (v0.17.0), however it's not yet fully supported in other pipelines like StableDiffusionKDiffusionPipeline
, this is probably under work anyway given this PR , but I'm adding a PR anyway as it is a really simple feature to add by simply changing
diffusers/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_k_diffusion.py
Line 48 in 790212f
class StableDiffusionKDiffusionPipeline(DiffusionPipeline, TextualInversionLoaderMixin): |
to
class StableDiffusionKDiffusionPipeline(DiffusionPipeline, TextualInversionLoaderMixin, LoraLoaderMixin):
I have also tested the same with my branch with the following code on a LoRA checkpoint (trained using kohya-ss trainer):
pipe = StableDiffusionKDiffusionPipeline.from_pretrained('/content/lora/training_scripts/stable-diffusion-v1-5',
torch_dtype=torch.float16,
safety_checker = None,
).to('cuda')
pipe.set_scheduler('sample_dpmpp_2m')
pipe.load_lora_weights(".", weight_name="</content/topdowobliquefurniture.safetensors")
pipe._lora_scale = 0.8
prompt = "stone castle, topdowobliquefurniture"
negative_prompt = ("(low quality, worst quality:1.4), (grainy:2.0)")
images = pipe(prompt=prompt,
negative_prompt=negative_prompt,
width=512,
height=512,
guidance_scale = 7.0,
num_inference_steps=15,
num_images_per_prompt=1,
use_karras_sigmas=True
# generator=torch.manual_seed(0)
).images
images[0]
Metadata
Metadata
Assignees
Labels
No labels