[nodes] Add latent nodes, storage, and fix iteration bugs #3091
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
damian0815
reviewed
Apr 1, 2023
damian0815
reviewed
Apr 1, 2023
Contributor
|
i paid most attention to |
Contributor
|
thanks for the |
Collaborator
|
Just be sure to set the cache directory correctly; otherwise we’ll end up
with the VAE stored in multiple locations.
On Sun, Apr 2, 2023 at 4:57 PM Damian Stewart ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In invokeai/app/invocations/latent.py
<#3091 (comment)>:
> + callback=step_callback
+ )
+
+ # https://discuss.huggingface.co/t/memory-usage-by-later-pipeline-stages/23699
+ torch.cuda.empty_cache()
+
+ name = f'{context.graph_execution_state_id}__{self.id}'
+ context.services.latents.set(name, result_latents)
+ return LatentOutput(
+ latent=LatentField(latent_name=name)
+ )
+
+
+# Latent to image
+class LatentToImageInvocation(BaseInvocation):
+ """Generates an image from a latent."""
@Kyle0654 <https://github.com/Kyle0654> according to
https://huggingface.co/stabilityai/sd-vae-ft-mse you should be able to do
something like:
from diffusers.models import AutoencoderKL
vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse")
—
Reply to this email directly, view it on GitHub
<#3091 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAA3EVNYE4Q7V56DEXPS3KTW7HR2PANCNFSM6AAAAAAWPKXMLU>
.
You are receiving this because you commented.Message ID:
***@***.***>
--
Written on my cell phone. Anything that seems odd is the fault of
auto-correct.
|
Contributor
Author
|
@lstein if we get this merged, could you later add something to model manager to get just the vae (in a cache-friendly way) and update LatentsToImage to use that? |
blessedcoolant
approved these changes
Apr 4, 2023
lstein
approved these changes
Apr 4, 2023
Collaborator
lstein
left a comment
lstein
left a comment
There was a problem hiding this comment.
I like the graph plotting in the CLI.
Collaborator
|
Sure. |
Add collection nodes, math nodes. Add noise node. Add some graph debug commands to the CLI. Fix negative id linking in CLI.
Fix a CLI bug with multiple links per node
Kyle0654
force-pushed
the
kyle0654/node_latents
branch
from
195fa70 to
cd03df9
Compare
Collaborator
The model manager part of this request is now in PR #3120. As soon as both this PR and that one are merged with |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds latent nodes for text to latent and latent to latent (roughly equivalent to text to image and image to image).
Adds latent storage.
Adds simple math nodes.
Fix bug in iteration expansion that was causing incorrect edges.
Add simple range and random range collection generator nodes.
Fix bug in CLI when linking to multiple nodes.
Example of new nodes:
Generates 3 different images.