Skip to content

[Examples] Fix type-casting issue in the ControlNet training script #2994

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Apr 12, 2023
Merged

[Examples] Fix type-casting issue in the ControlNet training script #2994

merged 5 commits into from
Apr 12, 2023

Conversation

sayakpaul
Copy link
Member

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Apr 6, 2023 •
edited
Loading

The documentation is not available anymore as the PR was closed or merged.

williamberman
williamberman approved these changes Apr 6, 2023
View reviewed changes
Copy link
Contributor

@williamberman williamberman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

a bit confused why it didn't happen before but should be safe to merge

@sayakpaul
Copy link
Member Author

Maybe the training runs were not using mixed precision at all? Can happen in the following scenario.

We likely have to specify the accelerate mixed_precision flag after accelerate launch --mixed_precision=fp16 ... as we use accelerator.mixed_precision to determine weight_dtype?

patrickvonplaten
patrickvonplaten approved these changes Apr 6, 2023
View reviewed changes
Copy link
Contributor

@patrickvonplaten patrickvonplaten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

@sayakpaul
Copy link
Member Author

I will wait for @williamberman to check out #2994 (comment) and ensure that we're on the same page regarding this.

@sayakpaul
Copy link
Member Author

@williamberman a friendly ping ^

@williamberman
Copy link
Contributor

We pass the mixed_precision arg from the cli args to the Accelerator object

diffusers/examples/controlnet/train_controlnet.py

Line 667 in 67ec9cf

mixed_precision=args.mixed_precision,

so I don't think it's an issue of whether or not accelerate launch was used. It's ok to merge without going super in depth here

@sayakpaul
Copy link
Member Author

@williamberman, hmm interesting!

I tried with an earlier accelerate version too and it failed without the changes introduced in this PR.

@sayakpaul sayakpaul merged commit e607a58 into main Apr 12, 2023
@sayakpaul sayakpaul deleted the fix/controlnet-training branch April 12, 2023 01:05
@sayakpaul sayakpaul mentioned this pull request Apr 12, 2023
Closed
w4ffl35 pushed a commit to w4ffl35/diffusers that referenced this pull request Apr 14, 2023
@sayakpaul @w4ffl35
[Examples] Fix type-casting issue in the ControlNet training script (h…
ce09ba3 
…uggingface#2994)

* fix: norm group test for UNet3D.

* fix: type-casting issue in controlnet training.
dg845 pushed a commit to dg845/diffusers that referenced this pull request May 6, 2023
@sayakpaul @dg845
[Examples] Fix type-casting issue in the ControlNet training script (h…
3f3ab51 
…uggingface#2994)

* fix: norm group test for UNet3D.

* fix: type-casting issue in controlnet training.
AmericanPresidentJimmyCarter pushed a commit to AmericanPresidentJimmyCarter/diffusers that referenced this pull request Apr 26, 2024
@sayakpaul
[Examples] Fix type-casting issue in the ControlNet training script (h…
82176c7 
…uggingface#2994)

* fix: norm group test for UNet3D.

* fix: type-casting issue in controlnet training.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yiyixuxu yiyixuxu Awaiting requested review from yiyixuxu

2 more reviewers

@williamberman williamberman williamberman approved these changes

@patrickvonplaten patrickvonplaten patrickvonplaten approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@sayakpaul @HuggingFaceDocBuilderDev @williamberman @patrickvonplaten