Skip to content

Add Dream 7B Diffusion Large Language Model Pipeline #12091

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 23 commits into
base: main
Choose a base branch
from
+2,420 −0
Open

Add Dream 7B Diffusion Large Language Model Pipeline #12091

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
362f10a
Initial implementation of Dream Transformer model
dg845 Aug 7, 2025
90ed142
make style and make quality
dg845 Aug 7, 2025
af38299
Initial implementation of Dream masked diffusion scheduler
dg845 Aug 11, 2025
ee6f585
make style and make quality
dg845 Aug 11, 2025
0b52b1f
Improve Dream transformer forward method and add embed_tokens method …
dg845 Aug 15, 2025
4b7ef3b
Add comment in scheduler further explaining shifting model_outputs
dg845 Aug 15, 2025
b7e6991
Initial commit for Dream 7B LLM pipeline
dg845 Aug 15, 2025
ed50d57
make style and make quality
dg845 Aug 15, 2025
94d3cf5
Merge branch 'main' into dream-7b-pipeline
dg845 Aug 15, 2025
506354e
Add transformers-style Dream tokenizer from the original code
dg845 Aug 20, 2025
733bde7
make style and make quality
dg845 Aug 20, 2025
a49fe39
Merge branch 'main' into dream-7b-pipeline
dg845 Aug 20, 2025
c1b1a27
Add DreamTokenizer to src/diffusers/__init__.py
dg845 Aug 20, 2025
89b868a
Force Dream RoPE freqs to be calculated in float32
dg845 Aug 21, 2025
ff03fc2
Fix shape errors in Dream transformer
dg845 Aug 21, 2025
c83abf5
Add modeling tests for the Dream transformer
dg845 Aug 22, 2025
52d3bfd
Make Dream scheduler conform to the scheduler public API
dg845 Aug 22, 2025
d2b85b0
Create dummy timestep schedule on init for compatibility with schedul…
dg845 Aug 22, 2025
1abd436
Add Dream masked diffusion scheduler tests
dg845 Aug 24, 2025
b61025a
Get Dream scheduler tests working
dg845 Aug 24, 2025
cae9ddf
Add DreamTextPipeline initial fast tests
dg845 Aug 26, 2025
c78d823
Fix bugs in DreamTextPipeline.__call__
dg845 Aug 26, 2025
879b41e
Change Dream scheduler so that self.timesteps has length num_inferenc…
dg845 Aug 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/diffusers/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,7 @@
"ControlNetXSAdapter",
"CosmosTransformer3DModel",
"DiTTransformer2DModel",
"DreamTransformer1DModel",
"EasyAnimateTransformer3DModel",
"FluxControlNetModel",
"FluxMultiControlNetModel",
Expand Down Expand Up @@ -301,6 +302,7 @@
"DPMSolverMultistepInverseScheduler",
"DPMSolverMultistepScheduler",
"DPMSolverSinglestepScheduler",
"DreamMaskedDiffusionScheduler",
"EDMDPMSolverMultistepScheduler",
"EDMEulerScheduler",
"EulerAncestralDiscreteScheduler",
Expand Down Expand Up @@ -413,6 +415,8 @@
"CosmosTextToWorldPipeline",
"CosmosVideoToWorldPipeline",
"CycleDiffusionPipeline",
"DreamTextPipeline",
"DreamTokenizer",
"EasyAnimateControlPipeline",
"EasyAnimateInpaintPipeline",
"EasyAnimatePipeline",
Expand Down Expand Up @@ -858,6 +862,7 @@
ControlNetXSAdapter,
CosmosTransformer3DModel,
DiTTransformer2DModel,
DreamTransformer1DModel,
EasyAnimateTransformer3DModel,
FluxControlNetModel,
FluxMultiControlNetModel,
Expand Down Expand Up @@ -957,6 +962,7 @@
DPMSolverMultistepInverseScheduler,
DPMSolverMultistepScheduler,
DPMSolverSinglestepScheduler,
DreamMaskedDiffusionScheduler,
EDMDPMSolverMultistepScheduler,
EDMEulerScheduler,
EulerAncestralDiscreteScheduler,
Expand Down Expand Up @@ -1048,6 +1054,8 @@
CosmosTextToWorldPipeline,
CosmosVideoToWorldPipeline,
CycleDiffusionPipeline,
DreamTextPipeline,
DreamTokenizer,
EasyAnimateControlPipeline,
EasyAnimateInpaintPipeline,
EasyAnimatePipeline,
Expand Down
2 changes: 2 additions & 0 deletions src/diffusers/models/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@
_import_structure["transformers.transformer_cogview3plus"] = ["CogView3PlusTransformer2DModel"]
_import_structure["transformers.transformer_cogview4"] = ["CogView4Transformer2DModel"]
_import_structure["transformers.transformer_cosmos"] = ["CosmosTransformer3DModel"]
_import_structure["transformers.transformer_dream"] = ["DreamTransformer1DModel"]
_import_structure["transformers.transformer_easyanimate"] = ["EasyAnimateTransformer3DModel"]
_import_structure["transformers.transformer_flux"] = ["FluxTransformer2DModel"]
_import_structure["transformers.transformer_hidream_image"] = ["HiDreamImageTransformer2DModel"]
Expand Down Expand Up @@ -165,6 +166,7 @@
ConsisIDTransformer3DModel,
CosmosTransformer3DModel,
DiTTransformer2DModel,
DreamTransformer1DModel,
DualTransformer2DModel,
EasyAnimateTransformer3DModel,
FluxTransformer2DModel,
Expand Down
1 change: 1 addition & 0 deletions src/diffusers/models/transformers/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
from .transformer_cogview3plus import CogView3PlusTransformer2DModel
from .transformer_cogview4 import CogView4Transformer2DModel
from .transformer_cosmos import CosmosTransformer3DModel
from .transformer_dream import DreamTransformer1DModel
from .transformer_easyanimate import EasyAnimateTransformer3DModel
from .transformer_flux import FluxTransformer2DModel
from .transformer_hidream_image import HiDreamImageTransformer2DModel
Expand Down
Loading