latents generation

[shengzhang90]

Hi, MAISI team

When I trained the maisi model with my own private data, the generation effect is as follow:

I doubt that the issue occurs at the latent generation, could you please help me? I will appreciate your help very much.

Best wishes

[Can-Zhao]

Could you help give more information on how you generated latents?
Did you use MAISI VAE or your own VAE?
How did you do image preprocessing especially intensity transform?

[Can-Zhao]

If 2D visualization is correct, but 3D is not, it means your 3D visualization method is incorrect.

[shengzhang90]

Did you visualize the 3D reconstruction result with the GT latents? Thanks a lot.

[Can-Zhao]

I'm a little bit confused. Yes I can see your visualized images, they look fine. What is your concern now?

[shengzhang90]

Finally, I solved the issue due to the easily overlooked and erroneous data type, I replace the np.uint16 by np.int16. Thanks a lot. HaHaHa!!!

[Can-Zhao]

Great!

[shengzhang90]

Hi, @Can-Zhao

When I finished the diffusion model training, I run the inference process, and the following is the generated result by scripts/diff_model_infer.py. My inference parameters are as follow:

{
"diffusion_unet_train": {
"batch_size": 1,
"cache_rate": 0,
"lr": 0.000004,
"n_epochs": 200,
"find_unused_parameters": true,
"random_seed": 0
},
"diffusion_unet_inference": {
"dim": [
256,
256,
128
],
"spacing": [
1.0,
1.0,
0.75
],
"top_region_index": [
0,
1,
0,
0
],
"bottom_region_index": [
0,
0,
1,
0
],
"random_seed": 0,
"num_inference_steps": 10,
"modality": 1
}
}

Could you please help me to have a look? Thanks in advance.