Skip to content

Conversation

@linoytsaban
Copy link
Collaborator

fixes the 3 issues with the advanced dreambooth lora sdxl training script, as described in #6486:

  1. redundant retraction of embeddings when performing pivotal tuning
  2. zeroing of text encoder params (when pivoting halfway we only optimize the embeddings during the first X% of steps - to handle correctly (accelerate wise) the param change midway for the optimizer, we set the relevant params to zero) only happens for one step
  3. incorrect casting of text encoder params to float32

1. redundant retraction
2. param clone
3. stopping optimization of text encoder params
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@linoytsaban linoytsaban marked this pull request as ready for review January 16, 2024 18:36
@linoytsaban linoytsaban self-assigned this Jan 17, 2024
Copy link
Member

@sayakpaul sayakpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge away!

Additionally, it'd be nice to put together a readme for the advanced diffusion training directory. This is because many of our users refer to them directly.

@sayakpaul sayakpaul merged commit 619e3ab into huggingface:main Jan 17, 2024
@linoytsaban
Copy link
Collaborator Author

definitely! Thanks! ❤️

@linoytsaban linoytsaban deleted the advanced_script_fixes branch February 8, 2024 17:07
AmericanPresidentJimmyCarter pushed a commit to AmericanPresidentJimmyCarter/diffusers that referenced this pull request Apr 26, 2024
…gingface#6486 (huggingface#6599)

* fixes bugs:
1. redundant retraction
2. param clone
3. stopping optimization of text encoder params

* param upscaling

* style
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants