diffusers DDIM gives worst results than Deforum DDIM

[AbdullahAlfaraj]

Describe the bug

diffusers DDIM gives worst results than Deforum DDIM when using small inference steps of size 8.
My understanding is that ddim is a very fast sampler, and
should yield a decent result even at a low step size like 8 or 16.
it's possible that I didn't set up the scheduler correctly. So, I've included the code below.

as for the deforum code you could try it on colab: https://colab.research.google.com/github/deforum/stable-diffusion/blob/main/Deforum_Stable_Diffusion.ipynb

Also, I've tried it with both CPU and GPU. it yields the same results.

Reproduction

import os
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
# make sure you're logged in with `huggingface-cli login`
from torch import autocast
from diffusers import StableDiffusionPipeline
from diffusers import DDIMScheduler

steps = 8
ddim = DDIMScheduler()
pipe = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", use_auth_token=True)
pipe = pipe.to("cpu")

prompt = "a photograph of an astronaut riding a horse"
# with autocast("cuda"):
    # image = pipe(prompt).images[0]  
image = pipe(prompt,num_inference_steps=steps,scheduler=ddim).images[0]  
image.save("output/output_{0}.png".format(1))

diffusers image outputs with scheduler set to ddim and steps = 8 :

Deforum with DDIM scheduler with steps = 8 :

Logs

No response

System Info

• diffusers version: 0.4.0.dev0
• Platform: Windows-10-10.0.19041-SP0
• Python version: 3.8.3
• PyTorch version (GPU?): 1.12.1 (False)
• Huggingface_hub version: 0.9.1
• Transformers version: 4.21.2
• Using GPU in script?: No
• Using distributed or parallel set-up in script?: No

[anton-l]

Hi @AbdullahAlfaraj! Because each diffusion model is trained using a specific noise schedule, the default parameters for DDIMScheduler() are not compatible with SD. To make it work, please refer to the pipeline's Readme:

scheduler = DDIMScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear", clip_sample=False, set_alpha_to_one=False)

pipe = StableDiffusionPipeline.from_pretrained(
    "CompVis/stable-diffusion-v1-4", 
    scheduler=scheduler,
    use_auth_token=True
).to("cuda")

[AbdullahAlfaraj]

thanks @anton-l , that seems to solve it.