Conversion Script: Update to Stable Diffusion 1.5 (EMA and non-EMA)

[redwraith2]

Since stable diffusion 1.5 was out, I was hoping to use it with the imagic colab. Could you upgrade diffusers to use the new model instead of the old one?

[samedii]

It's supported. Switch the model id to runwayml/stable-diffusion-v1-5

[patrickvonplaten]

Hey @redwraith2, it should work out of the box :-)

[InB4DevOps]

I tried to convert v1-5-pruned.ckpt (the 7gb version) to a diffuser model folder using scripts/convert_original_stable_diffusion_to_diffusers.py but the resulting model folder only has 4GB. The script also downloaded something which was only 4GB. Does the script need an update maybe?

[patrickvonplaten]

Hey @InB4DevOps,

The reason it only has 4GB is because we only extract the "non-ema" (not exponentially mean averaged) weights of the unet from the checkpoint and leave the "ema" weights - see:

diffusers/scripts/convert_original_stable_diffusion_to_diffusers.py

Line 295 in 2fb8faf

unet_key = "model.diffusion_model."

. But I agree that this is a bit badly documented at the moment! I'll open a PR :-)

[patrickvonplaten]

python scripts/convert_original_stable_diffusion_to_diffusers.py --checkpoint_path ../stable_diffusion_checkpoints/stable-diffusion-v1-5/v1-5-pruned.ckpt --dump_path ../ --scheduler_type lms --extract_ema

should now work with #991

[patrickvonplaten]

If you don't want the EMA weights you can just run:

python scripts/convert_original_stable_diffusion_to_diffusers.py --checkpoint_path ../stable_diffusion_checkpoints/stable-diffusion-v1-5/v1-5-pruned.ckpt --dump_path ../ --scheduler_type lms

[InB4DevOps]

Thank you @patrickvonplaten
Will give this a try later!

[toyxyz]

I have a similar problem. After the diffusers update, I converted SD v15.ckpt and then converted it back to cktp, but the file size became smaller. 7gb->4gb. And when converting ckpt files of different sizes with the convert_original_stable_diffusion_to_diffusers.py script, all results are the same size. is there something i am missing?

python convert_original_stable_diffusion_to_diffusers.py --checkpoint_path C:\ckpt\v1-5-pruned.ckpt --dump_path C:\ckpt\v15_2 --scheduler_type lms --extract_ema

python convert_diffusers_to_original_stable_diffusion.py --model_path C:\ckpt\v15_2 --checkpoint_path C:\ckpt\test_v15_2.ckpt

[patrickvonplaten]

The conversion should always go from 7GB to 4GB because either the EMA or non-EMA weights are used :-)

[toyxyz]

The conversion should always go from 7GB to 4GB because either the EMA or non-EMA weights are used :-)

Oh I understand! Which one is suitable for Dreambooth, Ema or Non Ema?

[patrickvonplaten]

Good question! I think the non-ema are better, but let's ask the experts here cc @patil-suraj @pcuenca :-)