ModuleNotFoundError: 'deprecated' when installing bokeh 3.0.1

[TimoRoth]

The full output is in our build output here:
https://github.com/OGGM/r2d/actions/runs/3449532207/jobs/5757527685#step:6:1589

Relevant part:

Collecting bokeh
  Downloading bokeh-3.0.1.tar.gz (15.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.5/15.5 MB 65.4 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [36 lines of output]
      DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
      Collecting setuptools>=64
        Using cached setuptools-65.5.1.tar.gz (2.6 MB)
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting setuptools-git-versioning
        Downloading setuptools-git-versioning-1.11.0.tar.gz (10 kB)
        Preparing metadata (setup.py): started
        Preparing metadata (setup.py): finished with status 'error'
        error: subprocess-exited-with-error
      
        × python setup.py egg_info did not run successfully.
        │ exit code: 1
        ╰─> [8 lines of output]
            Traceback (most recent call last):
              File "<string>", line 2, in <module>
              File "<pip-setuptools-caller>", line 34, in <module>
              File "/tmp/pip-install-fm8vdn8q/setuptools-git-versioning_7f0d0be05fad4a0f8518e6263fd1d55e/setup.py", line 3, in <module>
                from setuptools_git_versioning import version_from_git
              File "/tmp/pip-install-fm8vdn8q/setuptools-git-versioning_7f0d0be05fad4a0f8518e6263fd1d55e/setuptools_git_versioning.py", line 18, in <module>
                from deprecated import deprecated
            ModuleNotFoundError: No module named 'deprecated'
            [end of output]

All we're doing is trying to install bokeh. Is this an issue in their setup scripts, or an actual issue here?

[dolfinus]

Hi.

Please do not install this package from.tar.gz files, prefer .whl (binary) extension instead.

[TimoRoth]

That is not an option in this case, since I need to build a custom version of other binary dependencies, and in turn I have to use pip --no-binary.

[dolfinus]

Install setuptools-git-versioning from binary package, and then install bokeh with pip install --no-binary :all: --no-build-isolation to use installed package.

Or instead of --no-binary :all: pass package name which should be installed from source, like --no-binary bokeh
https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-no-binary

[TimoRoth]

On 12.11.2022 22:03, Maxim Martynov wrote: Install |setuptools-git-versioning| fron binary package, and then install bokeh with |pip install --no-binary :all: --no-build-isolation| to use installed package.

That'd cause a whole bunch of unnecessary build time only deps to be installed into our production container.

Or instead of |--no-binary :all:| pass package name which should be installed from source, like |--no-binary bokeh| https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-no-binary <https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-no-binary>

Then I would need to maintain a list of all packages bokeh (which in this case itself is just a dependency of something else) pulls in, and list all of them in --no-binary :all:. Why can you not instead correctly list build time dependencies for this project? Not doing so seems rather brittle to me anyway.

[dolfinus]

I removed publishing .tar.gz packages to PyPI because they are not meant to be used by anyone. 1.11.0 is the last version with .tar.gz distribution enabled, and if you're using --no-binary :all: pip downloads it. Latest release is 1.13.0, and it has proper dependencies list.

[dolfinus]

That'd cause a whole bunch of unnecessary build time only deps to be installed into our production container.

You can use multistage builds to create .whl for bokeh package, and then copy and install it in the production image.

[bryevdv]

@dolfinus FWIW I am 100% sympathetic to the idea that normal users should only ever use wheels. However, I have had to concede that there is a very small group of folks for whom sdists are in fact necessary, e.g people building packages for linux distributions, or users on specialized, uncommon platforms, and/or government or other institutions with legal or bureaucratic mandates to self-build for security considerations. I'm not sure if that is enough to make you re-consider, but I at least wanted to give context around what made me personally relent on this question for Bokeh.

[dolfinus]

Well, even if I'll publish sdist packages, it will not make much sense. setuptools-git-versioning uses current state of git repository to calculate the version number, but there is no git repository in .tar.gz file. What version number should be returned in this case?

[bryevdv]

That's an interesting question, I think setuptools-git-versioning is in a bit of an odd position here. But the Bokeh sdist already has a PKG-INFO with a version specified in it, so I don't actually think setuptools-git-versioning matters when building Bokeh from an sdist. Instead, it only matters when building the Bokeh sdist itself, from the Bokeh repo source. If there is a way to avoid listing setuptools-git-versioning in the Bokeh sdist, but use it when developing (i.e. in a repo), I guess that would be the real solution. But I am not aware of any way to draw out that distinction with current setuptools, pyproject, etc. (Please correct anything I might have wrong here, there's still lots of about Python packaging I find tortuous even after many, many years)

Edit: like, I guess it's possible the PKG-INFO inside the sdist is thrown away, and it does matter what setuptools-git-versioning say? I don't think so, but I could certainly be wrong.

[dolfinus]

The second issue here is setup.py of setuptools-git-versioning calls a function from the package itself to get version number. Due to this, package cannot be installed without --no-build-isolation flag because it should have access to its own source code (which is prevented by default).

[bryevdv]

So here is an experiment I conducted:

• downloaded bokeh-3.0.1.tar.gz
• untarred it
• manually removed setuptools-git-versioning from pyptoject.toml build dependencies
• tarred it back up
• ran pip install --no-binary :all: bokeh-3.0.1.tar.gz

That does build and install, but it does not work perfectly:

Installing collected packages: bokeh
  Attempting uninstall: bokeh
    Found existing installation: bokeh 3.0.1
    Uninstalling bokeh-3.0.1:
      Successfully uninstalled bokeh-3.0.1
Successfully installed bokeh-0.0.0

~/Downloads 2m 36s
310conda ❯ python
Python 3.10.6 (main, Oct 24 2022, 11:04:34) [Clang 12.0.0 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import bokeh
>>> bokeh.__version__
'0.0.0'
>>>

I guess the existing PKG-INFO in the sdist is not sufficient after all, which is unfortunate. (I would consider just patching our sdists to just remove setuptools-git-versioning from pyptoject.toml if it did).

I'm not really sure what can be done, then.

[dolfinus]

I need to think about this

[bryevdv]

🙏 thanks for looking into things @dolfinus

[bryevdv]

@dolfinus thinking a bit more about the question "What version number should be returned in this case?" — would it make sense, if there is a PKG-INFO present (i.e. if this is a build from an sdist), for setuptools-git-versioning to simply report that version?