Skip to content

Support views batch for panorama #3632

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Jun 6, 2023
Merged

Conversation

Isotr0py
Copy link
Contributor

@Isotr0py Isotr0py commented Jun 1, 2023

  • Support views batch in denosing loop for panorama pipeline

For P100 GPU, the speed was faster with view_batch_size = 8 than view_batch_size = 1

test code

import torch
from diffusers import StableDiffusionPanoramaPipeline, DPMSolverMultistepScheduler

view_batch_size = 8
seed = 86
model_ckpt = "stabilityai/stable-diffusion-2-base"
prompt = "a photo of the dolomites"

pipe = StableDiffusionPanoramaPipeline.from_pretrained(model_ckpt, torch_dtype=torch.float16).to("cuda")
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
pipe.to("cuda")
generator = torch.Generator(device="cuda").manual_seed(seed)
image = pipe(prompt, generator=generator, num_inference_steps=25, width=2048, height=512, view_batch_size=view_batch_size).images[0]
display(image)

del pipe
torch.cuda.empty_cache()

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Jun 1, 2023

The documentation is not available anymore as the PR was closed or merged.

@patrickvonplaten
Copy link
Contributor

cc @sayakpaul can you take a look?

@@ -474,6 +474,7 @@ def __call__(
width: Optional[int] = 2048,
num_inference_steps: int = 50,
guidance_scale: float = 7.5,
view_batch_size: int = 1,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also add an entry for this arg in the docstrings?

Copy link
Member

@sayakpaul sayakpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me! Thanks so much!

Some TODOs:

  • Add an entry for the new argument.
  • Mention this argument in the docs and comment on the potential speedup one might get.
  • Add a test to ensure robust behavior.

Let me know if anything is unclear.

Also cc: @omerbt

@Isotr0py Isotr0py requested a review from sayakpaul June 6, 2023 13:36
@@ -508,6 +510,9 @@ def __call__(
Paper](https://arxiv.org/pdf/2205.11487.pdf). Guidance scale is enabled by setting `guidance_scale >
1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`,
usually at the expense of lower image quality.
view_batch_size (`int`, *optional*, defaults to 1):
The batch size to denoise splited views. For some GPUs with high performance, higher view batch size
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The batch size to denoise splited views. For some GPUs with high performance, higher view batch size
The batch size to denoise split views. For some GPUs with high performance, higher view batch size

Copy link
Member

@sayakpaul sayakpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

I think there's just one thing pending -- mention this argument from the doc and comment about the potential speedups.

@patrickvonplaten
Copy link
Contributor

Good to merge for me whenever you want @sayakpaul

@omerbt
Copy link
Contributor

omerbt commented Jun 6, 2023

Thanks for the PR @Isotr0py, looks good!

@sayakpaul
Copy link
Member

Added a "tip" on view_batch_size in the doc. Will merge after the CI passes.

@sayakpaul sayakpaul merged commit 11b3002 into huggingface:main Jun 6, 2023
@Isotr0py Isotr0py deleted the panorama_batch branch June 7, 2023 03:18
yoonseokjin pushed a commit to yoonseokjin/diffusers that referenced this pull request Dec 25, 2023
* support views batch for panorama

* add entry for the new argument

* format entry for the new argument

* add view_batch_size test

* fix batch test and a boundary condition

* add more docstrings

* fix a typos

* fix typos

* add: entry to the doc about view_batch_size.

* Revert "add: entry to the doc about view_batch_size."

This reverts commit a36aeaa.

* add a tip on .

---------

Co-authored-by: Sayak Paul <[email protected]>
AmericanPresidentJimmyCarter pushed a commit to AmericanPresidentJimmyCarter/diffusers that referenced this pull request Apr 26, 2024
* support views batch for panorama

* add entry for the new argument

* format entry for the new argument

* add view_batch_size test

* fix batch test and a boundary condition

* add more docstrings

* fix a typos

* fix typos

* add: entry to the doc about view_batch_size.

* Revert "add: entry to the doc about view_batch_size."

This reverts commit a36aeaa.

* add a tip on .

---------

Co-authored-by: Sayak Paul <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants