[Docs] Doc Sprint on Wednesday Sep 7

[patrickvonplaten]

** There is still more TODO, to see open options, see this comment**

Let's add them docs.

• Create a minimum toctree with mostly blank pages to get it working
• Merge: https://github.com/huggingface/moon-landing/pull/3415
• Do a doc sprint right after

cc @anton-l

[patrickvonplaten]

Docs are now live 🎉

Thanks to @natolambert we have a rough template of diffusers docs: https://huggingface.co/docs/diffusers/index

In two days, we want to kick-start a doc-sprint where we add as many doc-strings as possible.
Here the rough draft: https://docs.google.com/document/d/1TwqMlQrjWXi5a0y-C-Wbzx70R4ZQP8f8mq-ARSqB5fI/edit?usp=sharing

Tomorrow we will share more details on what exactly what could be worked on.
Let us know if you'd like to participate in the doc-sprint on Wednesday with a 👍

[patrickvonplaten]

The general doc structure (which is defined here)
should look more or less as follows:

"1. Get started"
I think the following subsections for "1. Get started" would be nice:
"1.1" Diffusers (as is there already), but needs to be adapted
"1.2" Quicktour (as is there already), but needs to be adapted
“1.3” Installation

“2. Using Diffusers”
We only have vision at the moment so let’s maybe just focus on this:

2.1 Loading pretrained instances (explain from_pretrained for all pipelines, models, schedulers)

2.2 Pipelines for Inference
2.1.1 Unconditional Generation
2.1.2 Conditional Generation
2.1.3 Image2Image
2.1.4 In-painting
2.1.5 Create a custom pipeline

"3. Custom Hardware / Optimizing"

pipelines for inference (fp16, autocast, onnx, mps, OpenVino)

“4. Training Diffusers”

3.1 Unconditional Training
3.2 Text-inversion
3.3 Conditional Training (TODO)

“5. Conceptual guides”

"4.1" Here I'd maybe now make a whole section for "Stable Diffusion" with subsections like ethics, optimization, the Hub, ...
"4.2" Philosophy (as is there already), but would need to be adapted
"4.3" Contribution (we can more or less copy what we have here: )

“5. API”

This is pretty self-explanatory - all public methods need some nice docstring. For now I’d put all models (UNet2D, UNet2DCondition) in one doc string file and also put all schedulers in one doc string file. The each pipeline should get a “Pipeline card” just like we do for Transformers models -> we could just copy-paste the format from: https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines/stable_diffusion (which could be the stable_diffusion.mdx model card) to all other models.

[patrickvonplaten]

For section 5.

We need to adapt both the code files, e.g. add nice doc-strings to each public function and class as well as add some .mdx files for the main API classes.

In general section 5 will look as follows:

API
- Main classes 
  - Models (one file with info and docstrings about `ModelMixin`, `Unet2DModel` ,`Unet2DConditionModel`, `VQModel`, `AutoencoderKLModel` 
  - Schedulers (one file with info and docstring about all scheduler classes - cc @natolambert)
  - DiffusionPipeline (one file - only with `DiffusionPipeline` class)
  - Logging (can copy from transformers: https://huggingface.co/docs/transformers/main_classes/logging)
  - Config (one file - only with `ConfigMixin`) and `register_to_config`
  - Outputs (one file with all `BaseOutput` classes)
  
 - Pipelines (similar in spirit to Transformers models): https://huggingface.co/docs/transformers/model_doc/albert
  - Stable Diffusion (one file which can be a copy of: https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines/stable_diffusion)
  - DDPM (we should create a README similar to https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines/stable_diffusion

[patrickvonplaten]

Communication tomorrow will happen over a discord and we'll also hold a call at 9h30am Paris time.
Please make sure to join discord to join the call and to participate :-)

To join:
Follow this link https://discord.gg/bYA2G9yJtP, then click on the 🎨 emoji to get access to the diffusers group. There you'll find the "core-contributions" channel where we'll post all the important info!

[natolambert]

Working on scheduler updates in #376.

[patrickvonplaten]

For anybody who wants to join - ideally we start with 5. API docs for pipelines. Just pick one of the following pipelines and follow the PR: https://github.com/huggingface/diffusers/pull/377/files

• You should add a pipeline card, e.g. latent_diffusion.mdx
• add the file to the _toctree.yml: https://github.com/huggingface/diffusers/blob/main/docs/source/_toctree.yml
• Complete all pipeline file in your pipeline folder with correct docstrings in the code, e.g. https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/latent_diffusion/pipeline_latent_diffusion.py
  • Here make sure that at least the following functions and classes have a nice docstring: def __init__ and def __call__
  • Make sure to add the classes you want to expose to your pipeline card so that they are correctly displayed in the docs

Pick one of the following pipelines to get started 🚀 & please note below which pipeline you would like to work on with an open PR :-)

• 1. 1. ddim
• 2. 2. ddpm
• 3. 3. latent_diffusion
• 4. 4. latent_diffusion_uncond
• 5. 5. pndm
• 6. 6. score_sde_ve
• 7. 7. stable_diffusion
• 8. 8. stochatic_karras_ve

[patrickvonplaten]

Taking latent diffusion in this PR: https://github.com/huggingface/diffusers/pull/377/files

[pcuenca]

I can take stable diffusion if that's ok.

[kashif]

I can take 2 ddpm and pndm

[satpalsr]

I'd like to take latent_diffusion_uncond.