-
Notifications
You must be signed in to change notification settings - Fork 6.2k
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
The additional latents are random noise and instead need to be added to the original image as during inference
diffusers/examples/dreambooth/train_dreambooth.py
Lines 1215 to 1222 in 7a39691
if unet.config.in_channels > channels: | |
needed_additional_channels = unet.config.in_channels - channels | |
additional_latents = randn_tensor( | |
(bsz, needed_additional_channels, height, width), | |
device=noisy_model_input.device, | |
dtype=noisy_model_input.dtype, | |
) | |
noisy_model_input = torch.cat([additional_latents, noisy_model_input], dim=1) |
compared to
diffusers/src/diffusers/pipelines/deepfloyd_if/pipeline_if_superresolution.py
Lines 867 to 871 in 7a39691
upscaled = F.interpolate(image, (height, width), mode="bilinear", align_corners=True) | |
noise_level = torch.tensor([noise_level] * upscaled.shape[0], device=upscaled.device) | |
noise = randn_tensor(upscaled.shape, generator=generator, device=upscaled.device, dtype=upscaled.dtype) | |
upscaled = self.image_noising_scheduler.add_noise(upscaled, noise, timesteps=noise_level) |
Reproduction
n/a
Logs
No response
System Info
n/a
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working