Do we need python-dev?

[rahulpshah]

https://github.com/jupyter/docker-stacks/blob/master/minimal-notebook/Dockerfile#L23
Why do we need python-dev from the base image if we install python from conda?

[mathbunnyru]

And, python-dev is python2, not python3, which doesn't make sense at all. Could you please make a PR without this package, maybe it would be easy to not to install it?

[romainx]

Hello @rahulpshah,

You're right 😱

jovyan@e49d9de062c1:/usr/bin$ /usr/bin/python

# Python 2.7.17 (default, Apr 15 2020, 17:20:14) 
# [GCC 7.5.0] on linux2
# Type "help", "copyright", "credits" or "license" for more information.
# >>> 

python-dev installs a python version that is not needed.

jovyan@1238c0821be8:~$ apt-cache rdepends python-dev

# python-dev
# Reverse Depends:
#  python
#  python-minimal

We will reduce the image size and the same for all the downstream images

REPOSITORY                 TAG                 IMAGE ID            CREATED             SIZE    
jupyter/minimal-notebook   latest              bbcc28f4fde5        29 seconds ago      2.468GB
<none>                     <none>              ba8c39810ce2        11 days ago         2.514GB

So, as suggested by @mathbunnyru 👍 please could you make a PR?

Many thanks
Best

[rahulpshah]

Thanks for the response @mathbunnyru, @romainx. I will make the PR today. Are there any tests I need to verify the build?

[bollwyvl]

Recently got bitten by thinking i could actually bid a fond farewell to python2: so long as nodejs is supported (e.g. building jupyterlab, so, yes), python2 will likely have to be on the path, as it is part of the build chain for packages with native components.

[romainx]

Hello @bollwyvl,

Thanks for your feedback, it's interesting. I think all the stacks can be build without this package. However it 's a problem if we break something elsewhere by removing it.
So could you please develop and / or give an example of what you mean by saying "as it is part of the build chain for packages with native components"?

Many thanks for your help.
Best.

[bollwyvl]

Specifically nodejs packages. If there are any "jupyterlab build" commands in the stack, and any of the core lab npm components, any part of its build chain, or any extensions or their npm dependencies, require compiling with the weird node-gyp toolchain, the build will fail if python2 is not available. Ubuntu 20.04 and Fedora 32, for example, don't ship with python2 by default, and no doubt other distros will follow suit, and these will have to be installed by default. Since it's unlikely any of the stacks use conda to install a python2 command, this can result in hard, unpleasant fails, basically amounting to "node-gyp failed: no python2 command found". They would likely not require python-dev.

[rahulpshah]

Hi @bollwyvl Thanks for pointing that out. I checked out the node-gyp page it looks like the allow configuring the python path to be used for the node-gyp. Is it something that we can benefit from. I am thinking of adding a line here (https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile#L117) to set the config for node-gyp. Do you have an example that I can use to validate where the node-gyp needs the python2 binary.

[romainx]

Hello all,

In fact just figured out that python-minimal (Python 2.7) is installed despite the removal of the python-dev package. It's a dependency of texlive.

$ dpkg -S /usr/bin/python
# python-minimal: /usr/bin/python

$ apt-cache rdepends --installed python-minimal
# python-minimal
# Reverse Depends:
#  python

$ apt-cache rdepends --installed python
# python
# Reverse Depends:
#  texlive-pictures
#  texlive-latex-extra
#  python-minimal

And at the end it is installed by this line

docker-stacks/minimal-notebook/Dockerfile

Line 25 in 3b1f4f5

texlive-xetex \

So removing python-dev package will not remove python 2.7 but will only remove some additional packages.

ii  libpython2.7-dev:amd64      2.7.17-1~18.04ubuntu1               amd64        Header files and a static library for Python (v2.7)
ii  libpython-dev:amd64         2.7.15~rc1-1                        amd64        header files and a static library for Python (default)
ii  libpython2.7:amd64          2.7.17-1~18.04ubuntu1               amd64        Shared Python runtime library (version 2.7)
ii  python-dev                  2.7.15~rc1-1                        amd64        header files and a static library for Python (default)
ii  python2.7-dev               2.7.17-1~18.04ubuntu1               amd64        Header files and a static library for Python (v2.7)

[mathbunnyru]

I think it is still worth to delete non-direct dependencies.
Packages and their dependencies change, so it is quite possible that texlive will not require python2 in the future.