Figure out a way for Sphinx to deal with mime bundles

[choldgraf]

Something that we have run into in implementing an IPYNB parser, as well as in thinking through an execution/cacheing step, is how to deal with the outputs of running Jupyter cells. AKA, how to handle mime bundles.

The way that jupyter-sphinx does this is to hard-code rules that parse the bundle and convert those directly to docutils objects at parse time. This isn't ideal, because this throws away information about the bundle before the rendering step, which is when you'd want that extra "mime-bundle-like" information.

So we should figure out a way to do something like:

• Given the output of a Jupyter cell, find a way to either represent this in a docutils node or to represent a "pointer" in docutils that can be used to fetch the outputs later
• Instructions for how, at the builder (transform?) step, this mimebundle should be grabbed and then rendered in an opinionated fashion (e.g. HTML renderers would pull JS/HTML if it's in an output, while PDF renderers would just pull static representations of the outputs)

[akhmerov]

The way that jupyter-sphinx does this is to hard-code rules that parse the bundle and convert those directly to docutils objects at parse time. This isn't ideal, because this throws away information about the bundle before the rendering step, which is when you'd want that extra "mime-bundle-like" information.

I fully agree. This should certainly be changed on the jupyter-sphinx side.

[choldgraf]

A prototype of this is now implemented in this repository!

Here we implement a parser that, among other things, adds a CellOutputsBundleNode when it encounters any outputs: https://github.com/ExecutableBookProject/sphinx-notebook/blob/master/sphinx_notebook/parser.py

And here we implement a post-transform that will parse these nodes at the builder stage to decide what needs to be output: https://github.com/ExecutableBookProject/sphinx-notebook/blob/master/sphinx_notebook/transform.py

Would love to hear what @akhmerov thinks about it, where it should be improved etc. I am still not getting proper ipywidget-style outputs (though I am leveraging a lot of jupyter sphinx under the hood)...not sure what's up with that. Also, there's the question of if / how to upstream some of this to jupyter-sphinx...happy to discuss that too

[akhmerov]

I'd love to see as much of this as possible move upstream: a lot of the logic here is much more sound than what we have in jupyter-sphinx.

At a glance looking at the code, I expect that there will be a need for the users to control the handling of different mime types at least on the level of setting priorities. As a personal example, I encountered the need to support e.g. 'application/vnd.plotly.v1+json'.

[choldgraf]

@akhmerov definitely - the main goal here is to have a dictionary of "builder type -> mime priority" pairs. So I'd imagine users can update this with something like the following sphinx config arg:

sphinx_notebook_mime_preference = {
    "<builder_name>" = [list-of-mime-preferences]
}

and it'd update that dict.

as well as

sphinx_notebook_append_mime_preferences = {
    "<builder_name>" = [list-of-mime-preferences-to-append-to-what's-there]
}

[akhmerov]

Another question from a cursory read: I see that the images are inlined; isn't that bad for page load performance? In jupyter-sphinx we were discussing even moving in the opposite direction by loading the widget state as an external resource.

[choldgraf]

probably so :-) right now it's just a stop-gap because all of the nbconvert "FileOutputPreprocessor" stuff was confusing me ;-)

[akhmerov]

I have an educated guess why widgets don't work: widget views don't store anything. Rather all the data is dumped into a widget state, that we extract here. Having the widget state is required for the widgets to render.

EDIT: come to think of it, this is relevant info for execution and output cacheing. I'll open an issue there.

[choldgraf]

I gave a stab at fixing this in #10 though didn't quite get it working...if you have any thoughts there @akhmerov they'd be appreciated :-)