Enable xformers by default

[chinoll]

Is your feature request related to a problem? Please describe.
Diffusers are too slow to train in the default case.

Describe the solution you'd like
If xformers is installed in the environment, xformers is enabled by default.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[camenduru]

@chinoll good idea 🤩 I am doing like this with colab maybe helps

!pip install -qq https://github.com/camenduru/stable-diffusion-webui-colab/releases/download/0.0.14/xformers-0.0.14.dev0-cp37-cp37m-linux_x86_64.whl
!git clone https://github.com/huggingface/diffusers.git
!sed -i -e 's/if self._use_memory_efficient_attention_xformers:/if True:/g' /content/diffusers/src/diffusers/models/attention.py
!pip install -qq /content/diffusers

[patrickvonplaten]

I'm actually not opposed to enabling it by default in case it is correctly installed. So ok for me to maybe wrap enable_xformers_.. into a try ... except statement. cc @patil-suraj what do you think?

[patil-suraj]

Good idea, will open a for this soon :)

[patil-suraj]

Fixed in #1354

[jtoy]

what is the safe way to know if its enabled from code?

[patil-suraj]

Hey @jtoy , xformers attention is not enabled by default anymore see #1640, we need to call pipeline.enable_xformers_memory_efficient_attention() explicitly to enable it. Also right now there's no direct way to check if it's enabled.