Fix stable diffusion initial latents in the inpainting pipeline

[leszekhanusz]

Applying @nagolinc proposition in #261 to the inpainting pipeline.

Should fix #261

This modification allows to do outpainting without having a white border.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[patil-suraj]

Thanks a lot for the PR! Do you have a colab with some examples with which we can verify this ?

[leszekhanusz]

Thanks a lot for the PR! Do you have a colab with some examples with which we can verify this ?

I don't have a colab but it's easy to see it with diffusionui:

• Upload any image
• draw a big zone to inpaint (half bottom of image)
• inside that zone, paint everything with white
• set the strength to 1

You'll see a big difference whether the fix is applied or not.

Alternatively you could use the image and mask examples in the issue #261

[patrickvonplaten]

@patil-suraj @anton-l could we check this out maybe? Also related to: #328

[chengyjonathan]

I was playing around with this change, and I noticed that it would trigger this error more frequently:

if not mask.shape == init_latents.shape:
raise ValueError("The mask and init_image should be the same size!")

Even when I explicitly resize the images like so:

initial_image = init_image["image"].convert("RGB").resize((512, 512))
mask_image = init_image["mask"].convert("RGB").resize((512, 512))

[leszekhanusz]

@chengyjonathan could you please give us an example of image and mask which causes your problem?

[chengyjonathan]

It’s a little tricky to reproduce. Since I was passing the image and mask through a gradio interface.

…

On Sun, Sep 18, 2022 at 2:55 AM Hanusz Leszek ***@***.***> wrote: @chengyjonathan <https://github.com/chengyjonathan> could you please give us an example of image and mask which causes your problem? — Reply to this email directly, view it on GitHub <#469 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI254WMBS3FN4LEC3CO2GH3V63RJJANCNFSM6AAAAAAQJ2SU3Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.