i2vgen-xl implement

[zhw-zhang]

Model/Pipeline/Scheduler description

I2VGEN-XL recently disclosed their training and testing codes for Wensheng videos and Tusheng videos. Does diffusers have any plans to integrate these models and codes?
Code: https://github.com/damo-vilab/i2vgen-xl?tab=readme-ov-file

Open source status

• The model implementation is available.
• The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

No response

[shauray8]

the link does not work, here's a new link - https://github.com/damo-vilab/i2vgen-xl

[sayakpaul]

We're happy to accept contributions for this.

[a-r-r-o-w]

@sayakpaul I'd be happy to try and take this up since I noticed it hasn't been picked up for a while. It seems a little difficult from first glance but I'm sure I'll be able to figure a neat way to merge into diffusers.

[sayakpaul]

Cc: @DN6

[cocktailpeanut]

+1 Really looking forward to this. This is a huge pain point for all ModelScope models, because they are all hosted in China and it's near impossible to download the file from outside of China (For example I tried for 24 hours yesterday, just to download the 5GB model file and failed in the end). Wrote a quick thread about the severity of this issue today actually: https://x.com/cocktailpeanut/status/1745152494320112082?s=20

Would be great if diffusers can build this into it and serve from huggingface, so people from outside China could play with Modelscope models.

[sayakpaul]

On our mind.