Skip to content

ControlNet pipeline failed on mac M1 with "Assertion error: torch not compiled with cuda enabled" #2645

@RELNO

Description

@RELNO

Describe the bug

ControlNet pipeline failed on mac M1 with "Assertion error: torch not compiled with cuda enabled"

I've managed to follow the M1/M2 instructions to run baseline SD diffusers as described here: https://huggingface.co/docs/diffusers/optimization/mps

However, other pipelines failed with Assertion error: torch not compiled with cuda enabled. This is despite using device.to(mps)

Reproduction

from diffusers import StableDiffusionControlNetPipeline, ControlNetModel, UniPCMultistepScheduler
import torch

controlnet = ControlNetModel.from_pretrained(
    "lllyasviel/sd-controlnet-depth")
    
pipe = StableDiffusionControlNetPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5", 
    controlnet=controlnet) 
  
pipe = pipe.to("mps")
pipe.scheduler = UniPCMultistepScheduler.from_config(
    pipe.scheduler.config)

pipe.enable_model_cpu_offload()
controlnet = controlnet.to("mps")
generator = torch.Generator(device="cpu").manual_seed(user_seed)

# can't run on mps
# pipe.enable_xformers_memory_efficient_attention()

prompt = " Space station, pro photography, RAW photo, 8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3"
n_prompt = "text, watermark, blurry, not sharp, not accurate"
user_seed = -1

# open local image file and as init_image
from PIL import Image
init_img = Image.open("test.jpg")

_ = pipe(
    prompt,
    init_img,
    num_inference_steps=1
)

generator = torch.Generator(device="cpu").manual_seed(user_seed)
pipe = pipe.to("mps")
pipe.enable_attention_slicing()

image = pipe(
    prompt=prompt,
    negative_prompt=n_prompt,
    width=800,
    height=800,
    image=init_img,
    generator=generator,
    num_inference_steps=30
    ).images[0]

Logs

Output exceeds the size limit. Open the full output data in a text editor
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[9], line 1
----> 1 _ = pipe(
      2     prompt,
      3     init_img,
      4     num_inference_steps=1
      5 )

File ~/git/venv/mlr/lib/python3.9/site-packages/torch/autograd/grad_mode.py:27, in _DecoratorContextManager.__call__..decorate_context(*args, **kwargs)
     24 @functools.wraps(func)
     25 def decorate_context(*args, **kwargs):
     26     with self.clone():
---> 27         return func(*args, **kwargs)

File ~/git/venv/mlr/lib/python3.9/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_controlnet.py:697, in StableDiffusionControlNetPipeline.__call__(self, prompt, image, height, width, num_inference_steps, guidance_scale, negative_prompt, num_images_per_prompt, eta, generator, latents, prompt_embeds, negative_prompt_embeds, output_type, return_dict, callback, callback_steps, cross_attention_kwargs, controlnet_conditioning_scale)
    694 do_classifier_free_guidance = guidance_scale > 1.0
    696 # 3. Encode input prompt
--> 697 prompt_embeds = self._encode_prompt(
    698     prompt,
    699     device,
    700     num_images_per_prompt,
    701     do_classifier_free_guidance,
    702     negative_prompt,
    703     prompt_embeds=prompt_embeds,
...
    222 if _cudart is None:
    223     raise AssertionError(
    224         "libcudart functions unavailable. It looks like you have a broken build?")

AssertionError: Torch not compiled with CUDA enabled

System Info

  • diffusers version: 0.14.0
  • Platform: macOS-13.2.1-arm64-arm-64bit
  • Python version: 3.9.6
  • PyTorch version (GPU?): 1.13.1 (False)
  • Huggingface_hub version: 0.13.1
  • Transformers version: 4.26.1
  • Accelerate version: 0.17.0
  • xFormers version: not installed
  • Using GPU in script?: NO
  • Using distributed or parallel set-up in script?: NO

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingstaleIssues that haven't received updates

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions