[LoRA] Freezing the model weights #2245
Conversation
Freeze the model weights since we don't need to calculate grads for them.
|
The documentation is not available anymore as the PR was closed or merged. |
There was a problem hiding this comment.
This makes sense to me! @patil-suraj @williamberman could you also take a quick look?
| unet.requires_grad_(False) | ||
| vae.requires_grad_(False) | ||
|
|
||
| params = itertools.chain( |
There was a problem hiding this comment.
Why not just to text_encoder.requires_grad_(False) ?
There was a problem hiding this comment.
+1, , this way of freezing is only needed for the textual inversion
here we can use text_encoder.requires_grad_(False)
|
Also cc @sayakpaul - I think we can freeze everything here just like we do for dreambooth_lora |
+1 |
| unet.requires_grad_(False) | ||
| vae.requires_grad_(False) | ||
|
|
||
| params = itertools.chain( |
There was a problem hiding this comment.
+1, , this way of freezing is only needed for the textual inversion
here we can use text_encoder.requires_grad_(False)
Co-authored-by: Patrick von Platen <[email protected]>
|
Thanks a lot @erkams ! |
* [LoRA] Freezing the model weights Freeze the model weights since we don't need to calculate grads for them. * Apply suggestions from code review Co-authored-by: Patrick von Platen <[email protected]> * Apply suggestions from code review --------- Co-authored-by: Patrick von Platen <[email protected]> Co-authored-by: Suraj Patil <[email protected]>
As in original lora repository freezing the weights of the models helps us to use less memory