backends and multithreading

[aestrivex]

The current pysurfer code to use traitsui involves strange hacks to make the GUI busy for a short time while X renders the mayavi window but also without causing the GUI to enter an event loop and block. This section of code is very difficult to understand and maintain. A much more natural solution would be to use multithreading and explicit management of locking constructs to ensure that all of the events happen at the correct time.

The problem with this code is mainly with traitsui, which is an interactive GUI but which pysurfer is trying to use as a noninteractive GUI. Traitsui doesn't expose an API to usefully allow for precise timing of the GUI to be controlled in a multithreaded environment. In order to get the correct behavior without generating potential race conditions (which I think is what is happening now, only it is a race condition that is extremely unlikely to ever lose the race), I don't think embedding mayavi in traitsui will work.

I don't know any visualization libraries other than mayavi but I have been working on some simple tests to operate on VTK windows can be managed and display at the correct times in GUI code that does not involve traitsui -- I have proof of concept tests that suggest this should work with wxpython and tk (but it did not work at all with pyqt). As wxpython is not included in anaconda by default (and nothing about the GUI layout in pysurfer is particularly complex), tk is probably the better choice.

I'm of the opinion that to fix the current problems with traitsui, pysurfer should move in this direction (and I anticipate doing the majority of the work). This won't per se help eliminate dependencies on python 2.7 more quickly, but it will provide something more stable and maintainable so that pysurfer might later move to multiple backends as has been discussed before.

[larsoner]

I currently plan to migrate PySurfer backend to vispy in the next 3 months. Would that solve your use case?

[aestrivex]

I don't know anything about vispy, but no. Part of the motivation is that certain tasks in pysurfer that I haven't coded (like parsing curvature files) will work on brains in cvu which uses mayavi. If this is clearly counterproductive to pysurfer's goals, then I won't do it, but if it a matter of fixing and maintaining a mayavi backend alongside other backends, it is worthwhile.

[larsoner]

You're right that we could make it possible to allow the user to select whether mayavi/traits or vispy (or something else) is used as the backend. This has implementation disadvantages in terms of complexity and having to support two or more use cases, but it would have the advantage of being backward-compatible, and compatible with your use case or others that might currently depend on the mayavi-ness of PySurfer. @mwaskom @agramfort WDYT?

[mwaskom]

Is Mayavi ever going to move to Python 3?

Also what is cvu?

[aestrivex]

mayavi will move to python 3 eventually.

cvu is the connectome visualization utility, my program: https://github.com/aestrivex/cvu

[larsoner]

Last I heard (3-6 months ago?), the short answer was "not soon". The last comment on the Mayavi git page was 7 months ago (enthought/mayavi#84):

"The plans for Mayavi on Python 3 are linked to VTK being available on Python 3. The roadmap to get there is unclear. See http://www.vtk.org/Wiki/VTK/Python_Wrapping_FAQ."

So, first there's a wait for VTK to be ported, then Mayavi can be ported. There was also this response from a VTK dev about a year ago:

enthought/mayavi#84

This makes it sound like a push to port VTK will be more likely once OSX changes to Py3k. So if you're willing to wait for Apple to change their software so VTK can change their software so Mayavi can change their software, then yeah, it will be ported. But I am not too optimistic about the timeline...

[larsoner]

Although people are talking about making some progress, so maybe I'm just pessimistic:

swcarpentry/DEPRECATED-bc#107

[mwaskom]

What do you mean "like parsing curvature files"?

[aestrivex]

Pysurfer is able to read and display surface curvature. I have taken advantage of this before in cvu, to create visualizations that combine features from pysurfer and features written for cvu. Here is one example, which uses cvu to do the region definition for several regions, pysurfer to get the surface curvature, as well as various other things.

In most of the cases where I use functionality from both programs it wouldn't be hard to reproduce what pysurfer is doing. But if it is already written, why write it again?

[mwaskom]

I should point out that if you're just reading the curv files, that's really happening in nibabel now and in independent of display backend.

[mwaskom]

By the way your link doesn't work.

[larsoner]

@mwaskom it's just formatted backward -- the name of the link came out as the URL and vice-versa. Copy-paste the text and it should work.

[aestrivex]

Fixed the link, sorry.

I'm not entirely sure if the backend was relevant for this example. I have also done things like open pysurfer brains in cvu windows alongside cvu brains before, this just happened to be the instance I wrote up in my documentation.

[wmvanvliet]

@Eric89GXL any progress on the move to vispy? :)

[larsoner]

Heh, not much so far