[Feature] Latent format save / load with optional format conversion - using city96's SD-Latent-Interposer models #1181
Description
Feature Summary
Latent format save / load with optional format conversion - using city96's SD-Latent-Interposer models
Detailed Description
Here ( https://github.com/city96/SD-Latent-Interposer ) there are some Python code of a small neural net to convert between SD / SD3 / SDXL / FLUX latent formats. Here ( https://huggingface.co/city96/SD-Latent-Interposer/tree/main ) are the model weights in safetensors format. Maybe it will be good to generate image with sdcpp using some SDXL-model (with the tons of LoRAs available for it), but as the last step, save the latent-image instead of the PNG (via VAE/TAE). And in the next run of sdcpp, load the latent-image and continue the generation with SD3 model - which can "write" text on this image. I think it would be faster, than save as PNG, and in the next run, load the PNG as the init image. Eg:
Step 1:
sd-cli -m SDXL.gguf --save-latent X.ltnt --latent-convert-model xl-to-v3_interposer-v4.0.safetensors ...
Step 2:
sd-cli -m SD3.gguf --load-latent X.ltnt ....
Of course the latent conversion is optional, and it would be best if I could use it either at save time or in load time.)
Alternatives you considered
No response
Additional context
No response