Client needs to be able to send data for annotation to remote-server (MONAILabel server)

[che85]

Is your feature request related to a problem? Please describe.
MONAILabel V1 only supports server-sided hosting of data.

For clinical use, we want our research assistants to be able to use MONAILabel from individual computers. A workstation with a dedicated GPU was assigned to be the remote-server, receiving client requests for annotating data and returning the annotation to the client.

Describe the solution you'd like
Clients should be able to send images for automatic annotation to the MONAILabel server.

Additional context
JolleyLab/DeepHeart#1
JolleyLab/DeepHeart#2

[lassoan]

It would be great if this interface would be exactly the same as the AIAA server interface. It would allow the same clients (Slicer, OHIF, MITK, ...) to use either MONAILabel or AIAA server.

[SachidanandAlle]

There is little difference between aiaa and MONAILabel.. but MONAILabel supports apis to upload your own image..

In your client.. you can follow this to create similar behavior

1. Upload image and get the id for further use.. /datastore api is helpful here..
2. Run inference and pass the image id from above step..

The current MONAILabel plugin in slicer fetches the images based on active learning strategies.. thats one reason it is tagged as active learning solution.. thats one important difference between aiaa (infer) only solution

[SachidanandAlle]

Let me add a quick option to push the image to server from client box.. this should unblock for users sitting remote.. and use their own samples to run infer kind of tasks..

[che85]

Ideal for our use case would be:

1. user loads image
2. user creates annulus contour curve
3. user creates landmarks
4. user initiates automatic segmentation process
   a. MONAILabel-client pre-processes data (normalization, orientation, resampling)
   b. MONAILabel-client uploads image + annulus contour (currently as a label map)
   c. MONAILabel-server runs inference and returns segmentation (multi-label)
5. user can refine segmentation

[che85]

How is additional information from the user transferred to the MONAILabel server? Are just coordinates of background/foreground landmarks sent via REST API?

[SachidanandAlle]

How is additional information from the user transferred to the MONAILabel server? Are just coordinates of background/foreground landmarks sent via REST API?

You can pass anything in params.. deepgrow sends some foreground/background points and uses the same during pre transform to prepare the input for the model/network

[lassoan]

@SachidanandAlle #156 looks awesome, thank you for implementing so quickly! @che85 could you test it?

For the leaflet segmentation the inputs are: an image, a closed curve, and a fiducial list with a few named points. @SachidanandAlle how do you think these could be added to the MONAILabel module GUI? A few options:

1. Command-line interface GUI generation

Slicer supports generating user interface for command-line executables using an XML file. The file lists all input and output parameters of an algorithm. It is used in MITK, MeVisLab, MedInria, Gimias, and a number of other applications - see more information here),

See SEM interface XML files in each subfolder here:
https://github.com/Slicer/Slicer/tree/master/Modules/CLI

For example, this description file:
https://github.com/Slicer/Slicer/blob/master/Modules/CLI/ACPCTransform/ACPCTransform.xml

Generates this GUI:

Slicer also writes all the selected data objects to files and simple parameters (Booleans, floats, etc.) into a command-line argument list. This could be adapted to send parameters via network.

2. Qt Designer .ui file

The MONAILabel app could send a Qt designer .ui file that Slicer can display in a GUI section. We could implement automatic sending of GUI content to the server (e.g., if the GUI element is a slider then we would send a number, if the GUI element is a node selector then we would write that node to file and send that file).

2. Python code

The client could send a Python class to the MONAILabel Slicer module, which would contain a few methods, such as 1. setting up GUI widgets, and 2. sending data that the user specified in the widgets to the server. Running Python code received via the network is a bit higher security risk than just interpreting XML descriptor files, so maybe this option is not desirable in the long term.

[SachidanandAlle]

@che85
For the ideal usecase as you mentioned above, now you can push a volume to server so that you can run infer task (pre/inference/post transforms)

I believe you guys are expert in slicer, so you should be able to quickly add additional interactions that you are looking in MONAILabel plugin which is kind of specific to an interaction model you are building (using annulus contour curve and landing marks)

For a quick implementation (for project week), I suggest you can clone the main branch and add your interaction logic over there...

Once you have the required contour curve and landing marks you can send it in params (JSON/Dict) and access the same in pre-transforms. If you are looking to send binary data then you can either make use of label field and access the same in your pre-transforms

For example: Scribbles is one your kind where some other researcher from KCL tried to add extra interactions as part of the user-case
https://github.com/Project-MONAI/MONAILabel/tree/scribbles/sample-apps/segmentation_spleen_postproc

@lassoan we can discuss how to make it flexible for others to add/inject new interactions into MONAILabel plugin. This is where I need your help.

May be we can discuss ideas you suggested and the priority and importance of adding SegmentEditEffects kind of extensions for MONAILabel plugin... or how to import those bunch of standard interactions in MONAILabel plugin.

[SachidanandAlle]

How is additional information from the user transferred to the MONAILabel server? Are just coordinates of background/foreground landmarks sent via REST API?

in the base version we support only auto-segmentation and deepgrow models. First one doesn't need any input.. and second one needs foreground/background clicks from user

[lassoan]

@SachidanandAlle these are very good suggestions. I would be very happy to discuss about them. Would you be available to talk today or Thursday afternoon (Eastern time) on the Slicer project week discord?

[che85]

I would be interested in joining that call as well.