[Latent Upscaling] Restore noise #3018
Description
What API design would you like to have changed or added to the library? Why?
The StableDiffusionLatentUpscalePipeline should have the noise_level parameter reinstated. It was removed based on the false assumption that noise is undesirable. The author of the original commit said that:
this step [noise] theoretically can make the model work better on out-of-distribution inputs, but mostly just seems to make it match the input less, so it's turned off by default.
It is desirable for the input to undergo some changes to enhance the output quality. Additionally, the diffuser's API ought to offer developers complete control. It is impossible to anticipate all potential scenarios, hence all features should be made accessible.
What use case would this enable or better enable? Can you give us a code example?
This feature is crucial in obtaining superior outcomes from the latent upscaler, and its significance is highlighted by its inclusion as a prominent setting on AUTOMATIC1111's home screen. The setting is labeled as "Denoising strength" and is enabled by default to introduce noise. It is accessible by clicking on "High res fix," which I believe employs stabilityai/sd-x2-latent-upscaler.
The introduction of noise enables the latent upscaler to incorporate additional details that it would have otherwise overlooked.
By way of illustration, in the image below, the inclusion of noise has not only upscaled it but also considerably augmented its quality. I find that the girl's teeth, hair, and eyes are particularly prominent.
