Enable text encoder training in Dreambooth

[shirayu]

This PR enables text encoder training in Dreambooth.
It enhances learning results.

Reference

• A blog post in Japanese
  • Note: They state clearly in Japanese that the attached train_dreambooth_mod.py is licensed under the same license to the original license. (i.e. Apache-2.0 license)
• A twitter post in Japanese
  • Effect of text encoder training
  • The first image is done without text encoding training, while the second one is done with text encoding training.

Acknowledgments

This PR is based on Kohya S.'s useful reports.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[patrickvonplaten]

Yes this seems to help quite a bit no @patil-suraj - I'll let you review here :-)

[patil-suraj]

Thanks a lot the PR @shirayu , we also ran quite a few experiments, and found that training text encoder indeed helps a lot. We are writing a report on this and will publish soon https://wandb.ai/psuraj/dreambooth/reports/-WIP-Dreambooth-training-analysis--VmlldzoyNzk0NDc3 (don't share it yet :) )

However, it's a bit tricky to have this in the script and requires some additional changes for it to work correctly in distributed setting. I had already done this so opened a PR here #883. Will include you as co-author in that PR :)

Hope it's alright with you :)

[shirayu]

Great!
Thank you for the experiments and your concern.
It's alright for me, so I close this now.