Text to audio with default code produces an unplayable file

[tin2tin]

Describe the bug

Great with the new Text to audio feature. However, when I use the default script, the resulting file doesn't contain any audio, there are no errors printed in the console:

The file properties looks ok:

However, VLC can't ID the codec:

Here's the file:
Techno music with a strong_ upbeat tempo and high melodic riffs.zip

Reproduction

Just run the default script and check the resulting file.

Logs

No info in logs.

System Info

Diffusers 0.15
PyTorch 2.0.0
Win 11

[patrickvonplaten]

Hey @tin2tin,

Sorry could you copy-paste the exact command you ran? I'm not 100% sure what model and what code you used? I assume it's https://huggingface.co/cvssp/audioldm

[tin2tin]

It's the code from here: https://github.com/huggingface/diffusers/releases/tag/v0.15.0

from diffusers import AudioLDMPipeline
import torch

repo_id = "cvssp/audioldm"
pipe = AudioLDMPipeline.from_pretrained(repo_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

prompt = "Techno music with a strong, upbeat tempo and high melodic riffs"
audio = pipe(prompt, num_inference_steps=10, audio_length_in_s=5.0).audios[0]

import scipy

scipy.io.wavfile.write("techno.wav", rate=16000, data=audio)

[tin2tin]

@sanchit-gandhi Could you check if this is working for you? Could the error be on my end?

[heury]

I got a same error. Could you guide me on how to resolve it? I tested it with original AudioLDM code and it worked fine.

[devxpy]

It's the code from here: v0.15.0 (release)

from diffusers import AudioLDMPipeline
import torch

repo_id = "cvssp/audioldm"
pipe = AudioLDMPipeline.from_pretrained(repo_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

prompt = "Techno music with a strong, upbeat tempo and high melodic riffs"
audio = pipe(prompt, num_inference_steps=10, audio_length_in_s=5.0).audios[0]

import scipy

scipy.io.wavfile.write("techno.wav", rate=16000, data=audio)

Remove the torch_dtype=torch.float16 param, and it seems to work

[tin2tin]

@devxpy Yes, you're right. That solves the problem. So, the documentation needs to be updated.