Diffusers 0.26 break a lot of existing pipelines and 3rd party modules

[vladmandic]

Describe the bug

diffusers==0.26 via pr #6665 which introduces support for I2VGenXL
redefines commonly used params in BasicTransformerBlock

for example, if self.use_ada_layer_norm now becomes if norm_type == "ada_norm": which would mean that every single pipeline that relies on those layers would need to be updated.

note that those layers are not used only within diffusers, they are pretty standardized (and were introduced in diffusers because of that in timeframe of diffusers==0.14) and used by a lot of 3rd party code as well.
for example of a very popular 3rd party lib that relies on it for a long time: https://github.com/dbolya/tomesd

Reproduction

at the very least, this completely breaks StableDiffusionReferencePipeline and StableDiffusionXLReferencePipeline

Logs

AttributeError: 'BasicTransformerBlock' object has no attribute 'use_ada_layer_norm'

System Info

diffusers==0.26.1

Who can help?

@yiyixuxu @sayakpaul @DN6 @patrickvonplaten

[vladmandic]

reproduction code:

import torch
import diffusers
import diffusers.utils
cls = diffusers.utils.get_class_from_dynamic_module('stable_diffusion_reference', module_file='pipeline.py')
pipe = cls.from_single_file('/mnt/d/Models/stable-diffusion/sd15/lyriel-v16.safetensors', torch_dtype=torch.float16).to('cuda')
print(pipe.__class__)
ref_image = diffusers.utils.load_image("https://hf.co/datasets/huggingface/documentation-images/resolve/main/diffusers/input_image_vermeer.png")
output = pipe(ref_image=ref_image, prompt="test", num_inference_steps=10, reference_attn=True, reference_adain=True)
print(output)

[yiyixuxu]

@vladmandic

ah, thanks for reporting this!
I had no idea the transformer block is used that way!

diffusers/examples/community/stable_diffusion_reference.py

Line 377 in 13001ee

if self.use_ada_layer_norm:

Is it ok we update these community pipelines instead?

[vladmandic]

like i said, its also used by a lot of 3rd party libraries as well (i gave an example of one such really popular lib)
those are quite (sort-of) standardized params.

[yiyixuxu]

@vladmandic
ohh thanks!
ok - we will update our basic tranformer block and deprecate these params instead

[sayakpaul]

Happy to do a PR on Monday to deprecate the booleans flags in a more robust way. We should then do a patch release.

[yiyixuxu]

@sayakpaul
I added them back here #6841 Let's merge in quickly if it looks ok to you

I did not deprecate here cause I am not sure what's the best way - I think it will be a little bit annoying if it throws a warning every time the class is created, but it is what it is, I guess. Let me know what you think! we can do that in a separate PR

[sayakpaul]

Makes sense. Once #6841 is merged, I will take care of a patch release.