How can I use `*.safetensors` modles in `diffusers` library?

[Yiximail]

I found a really great model a few days ago. Then I wanted to use it on a pure API server (without webui or other Gradio interface)
Here is the model's path in huggingface: WarriorMama777/OrangeMixs

And I mainly refer to these two demos:
conditional_image_generation
stable-diffusion-v1-5

from diffusers import DiffusionPipeline
# I also tried it in StableDiffusionPipeline

pipe  = DiffusionPipeline.from_pretrained("WarriorMama777/OrangeMixs")
pipe = pipe.to("cuda")

Got an error below.

OSError: /root/.cache/huggingface/diffusers/models--WarriorMama777--OrangeMixs/snapshots/641d5be1a5f89a040e58f769cac02b328a277467 does not appear to have a file named config.json. Checkout 'https://huggingface.co//root/.cache/huggingface/diffusers/models--WarriorMama777--OrangeMixs/snapshots/641d5be1a5f89a040e58f769cac02b328a277467/None' for available files.

Then I tried to download the model first and then load the model path directly.
According to this documentation, from_pretrained should be able to accept the model path.

from diffusers import DiffusionPipeline
pipe = DiffusionPipeline.from_pretrained("/content/AbyssOrangeMix2_nsfw.safetensors")

Error again.

OSError: It looks like the config file at '/content/AbyssOrangeMix2_nsfw.safetensors' is not a valid JSON file.

After comparing runwayml/stable-diffusion-v1-5 with WarriorMama777/OrangeMixs.
I found that there are multiple folders in the root of stable-diffusion-v1-5: unet, vae, tokenizer...
After reviewing many tutorials and documents, I think these are probably what diffusers load, not *.ckpt or *.safetensors.

Then I looked up how to convert them. And I found diffusers has a script.
convert_original_stable_diffusion_to_diffusers

!git clone https://github.com/huggingface/diffusers.git

!python /content/diffusers/scripts/convert_original_stable_diffusion_to_diffusers.py \
 --checkpoint_path /content/AbyssOrangeMix2_nsfw.safetensors \
 --from_safetensors --dump_path /content/model

Traceback (most recent call last):
  File "/content/diffusers/scripts/convert_original_stable_diffusion_to_diffusers.py", line 19, in <module>
    from diffusers.pipelines.stable_diffusion.convert_from_ckpt import load_pipeline_from_original_stable_diffusion_ckpt
ModuleNotFoundError: No module named 'diffusers.pipelines.stable_diffusion.convert_from_ckpt'

Here is my test colab notebook: colab

I am not familiar with python and may ask some basic questions. I have been trying to solve it for the past two days but I still haven't found a solution, so I apologize for any uncomfortable. I would greatly appreciate your help.

[patrickvonplaten]

Hey @Yiximail,

Yeah the models of this repo are not at all in the correct format, converted all of them here: https://huggingface.co/OrangeMix

Note that if you'd like to save those under the safetensors format, you can just load and save them, e.g.

from diffusers import DiffusionPipeline

pipeline = DiffusionPipeline.from_pretrained("OrangeMix/AbyssOrangeMix2")
pipeline.save_pretrained("local_path", safe_serialization=True)

cc @Narsil we should maybe add a doc for safetensors :-)

[Narsil]

On it: #2122 :)

[Yiximail]

Thank you for explaining to me, and the conversion of the model.