Unable to find release for package cairocffi of version file-.cairocffi-VERSION

[majewsky]

I'm trying to nixify https://github.com/taigaio/taiga-back with pypi2nix. I applied the following patch to work around Asana/python-asana#80:

diff --git a/requirements.txt b/requirements.txt
index 9aaf3ebb61..e754ea4e0c 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,6 +1,6 @@
 -i https://pypi.org/simple
 amqp==2.4.2
-asana==0.6.7
+asana==0.8.2
 asn1crypto==0.24.0
 billiard==3.5.0.5
 bleach==2.1.4

Now I'm getting this error:

$ pypi2nix -V 3.6 -r requirements.txt -E postgresql_11 -E "libxml2 libxslt zlib gcc libffi openssl pkgconfig zlib libjpeg openjpeg libtiff freetype lcms2 libwebp tcl" --default-overrides
pypi2nix v1.8.1 running ...

Stage1: Downloading wheels and creating wheelhouse ...
Stage2: Extracting metadata from pypi.python.org ...
WARNING: Couldn't recognize license `Standard PIL License` for `Pillow`
WARNING: Couldn't recognize license `Copyright 2013 Alexey Kachayev` for `fn`
|-> from webencodings-0.5.1.dist-info|-> from webcolors-1.7.dist-info|-> from vine-1.3.0.dist-info|-> from urllib3-1.24.2.dist-info|-> from Unidecode-0.4.20.dist-info|-> from tinycss-0.4.dist-info|-> from six-1.12.0.dist-info|-> from setuptools-41.0.1.dist-info|-> from serpy-0.1.1.dist-info|-> from sampledata-0.3.7.dist-info|-> from requests_oauthlib-0.8.0.dist-info|-> from requests-2.21.0.dist-info|-> from redis-2.10.5.dist-info|-> from raven-6.1.0.dist-info|-> from pytz-2019.1.dist-info|-> from python_magic-0.4.13.dist-info|-> from python_dateutil-2.7.5.dist-info|-> from PyJWT-1.7.1.dist-info|-> from Pygments-2.2.0.dist-info|-> from pycparser-2.19.dist-info|-> from psycopg2_binary-2.7.5.dist-info|-> from psd_tools-1.4.dist-info|-> from premailer-3.0.1.dist-info|-> from Pillow-4.1.1.dist-info|-> from olefile-0.46.dist-info|-> from oauthlib-3.0.1.dist-info|-> from netaddr-0.7.19.dist-info|-> from MarkupSafe-1.1.1.dist-info|-> from Markdown-3.0.1.dist-info|-> from lxml-4.3.3.dist-info|-> from kombu-4.0.2.dist-info|-> from Jinja2-2.10.1.dist-info|-> from idna-2.8.dist-info|-> from html5lib-1.0.1.dist-info|-> from gunicorn-19.7.1.dist-info|-> from fn-0.4.3.dist-info|-> from easy_thumbnails-2.4.1.dist-info|-> from docopt-0.6.2.dist-info|-> from djmail-1.0.1.dist-info|-> from django_sr-0.0.4.dist-info|-> from django_sites-0.9.dist-info|-> from django_sampledatahelper-0.4.1.dist-info|-> from django_picklefield-0.3.2.dist-info|-> from django_pglocks-1.0.2.dist-info|-> from django_jinja-2.3.1.dist-info|-> from django_ipware-1.1.6.dist-info|-> from Django-1.11.20.dist-info|-> from diff_match_patch-20121119.dist-info|-> from cssutils-1.0.2.dist-info|-> from cssselect-1.0.3.dist-info|-> from cryptography-2.3.1.dist-info|-> from contextlib2-0.5.5.dist-info|-> from chardet-3.0.4.dist-info|-> from cffi-1.12.3.dist-info|-> from certifi-2019.3.9.dist-info|-> from celery-4.0.2.dist-info|-> from CairoSVG-2.0.3.dist-info|-> from cairocffi-file_.cairocffi_VERSION.dist-info
Error: Unable to find release for package cairocffi of version file-.cairocffi-VERSION

I'm not sure what it's doing there with the cairocffi version. It should be:

$ grep cairocffi requirements.txt
cairocffi==1.0.2

Any ideas? I'm using pypi2nix 1.8.1 from the NixOS 19.03 channel.

[seppeljordan]

Hi @majewsky,

thank you for reporting this issue. Would you mind trying to generate a package set with the current version of pypi2nix on master? You can simply install it on you local NixOS system via the following command:

nix-env -if https://github.com/nix-community/pypi2nix/archive/master.tar.gz

I would appreciate hearing back from you how that went.

[majewsky]

Thanks for the pointer. I installed master as of a580289, but unfortunately this gives a rather short output:

$ pypi2nix -V 3.6 -r requirements.txt -E postgresql_11 -E "libxml2 libxslt zlib gcc libffi openssl pkgconfig zlib libjpeg openjpeg libtiff freetype lcms2 libwebp tcl" --default-overrides
pypi2nix v2.0.0
 running ...

Stage1: Downloading wheels and creating wheelhouse ...
Do you want to report above issue (a browser will open with prefilled details of issue)? [y/N]: n
Error: While trying to run the command something went wrong.

I couldn't say yes to the question because this is running on a headless server. (I don't have a desktop machine with NixOS at hand.)

[seppeljordan]

Hey, would you mind rerunning the command with '-vvv' option?

[majewsky]

That's quite a bit of output, so I put it in a gist: https://gist.github.com/majewsky/7a0fca4c4a56d14b37a13c6a9bd82622

Quoting the significant parts:

    Collecting serpy==0.1.1 (from -r /tmp/pypi2nix/a36c924ace01823a41f62200fb23b1b2/5b4c1ab4ca4e40b00f4b597244ba5871.txt (line 55))
      Downloading https://files.pythonhosted.org/packages/6e/40/957005bc5f7996dcbf0ceedb302c4dc58fb9d2cceb5e2c3eb8a7f3c163e4/serpy-0.1.1.tar.gz
      Saved /tmp/pypi2nix/a36c924ace01823a41f62200fb23b1b2/download/serpy-0.1.1.tar.gz
        ERROR: Complete output from command python setup.py egg_info:
        ERROR: Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "/tmp/pypi2nix/a36c924ace01823a41f62200fb23b1b2/build/serpy/setup.py", line 4, in <module>
            import serpy
          File "/tmp/pypi2nix/a36c924ace01823a41f62200fb23b1b2/build/serpy/serpy/__init__.py", line 1, in <module>
            from serpy.fields import (
          File "/tmp/pypi2nix/a36c924ace01823a41f62200fb23b1b2/build/serpy/serpy/fields.py", line 1, in <module>
            import six
        ModuleNotFoundError: No module named 'six'
        ----------------------------------------
    ERROR: Command "python setup.py egg_info" failed with error code 1 in /tmp/pypi2nix/a36c924ace01823a41f62200fb23b1b2/build/serpy/

six is referenced in the requirements.txt, but only after serpy because of alphabetical order.

[seppeljordan]

So I tested some things and came to the following conclusions:

1. There is a bug in the setup.py script in serpy == 0.1.1. The setup.py script needs the six module to execute which is not installed in the pypi2nix environment. This means that the setup.py script exits with an import error as seen above. I would consider this a bug, but normally one could work around this with installing six in a previous step like so:

pip install six
pip install serpy==0.1.1

2. The current master version of pypi2nix has no way of handling these cases. It used to be possible to prepare the installation environment of pypi2nix to include python packages necessary for the setup of other packages via the -s of --setup-requires flag. Although this flag is still present it does not work as described in point 1. Besides this point, you can try the follwing workaround:

pypi2nix -V 3.6 -r requirements.txt -E postgresql_11 -E "libxml2 libxslt zlib gcc libffi openssl pkgconfig zlib libjpeg openjpeg libtiff freetype lcms2 libwebp tcl" --default-overrides -s six==1.12.0

Notice the -s six==1.12.0 in the end!

EDIT: servy -> serpy

EDIT: Possible workaround!

[majewsky]

As you expected, adding -s six==1.12.0 doesn't seem to change anything. Anyway, I added the output to the gist for reference (https://gist.github.com/majewsky/7a0fca4c4a56d14b37a13c6a9bd82622#file-pypi2nix-2-out). It's way shorter than the previous run, but that's probably because of all the derivations that were already built in the previous run.

[seppeljordan]

I cannot promise anything but I will try to get the -s flag working as intended again soon. This issue will be updated as soon as progress has been made.

[seppeljordan]

The proposed workaround should now work with the master branch version of pypi2nix and I think the -s six==1.12.0 option can even be omitted. Please reopen if the issue persists for you.

[majewsky]

Yes, updating to current master made it work for me. Thanks!

[majewsky]

Actually, I cannot get it to work just yet. I got the requirements.nix and requirements_override.nix (which you can find in this gist), but trying to import them in my derivation results in this error:

$ nix eval '(import ./requirements.nix {})' --show-trace
error: while evaluating the attribute 'interpreter' at /x/src/github.com/majewsky/nixos-modules/pkgs/taiga-backend/requirements.nix:76:14:
while evaluating the attribute 'outPath' at /nix/store/6xrz725n08v8akdh90l8lh7bg3imcz56-nixos-19.03.173251.56d94c8c69f/nixos/lib/customisation.nix:150:7:
while evaluating the attribute 'outPath' at /nix/store/6xrz725n08v8akdh90l8lh7bg3imcz56-nixos-19.03.173251.56d94c8c69f/nixos/lib/customisation.nix:150:7:
while evaluating the attribute 'buildCommand' of the derivation 'python3.6-python36-interpreter' at /nix/store/6xrz725n08v8akdh90l8lh7bg3imcz56-nixos-19.03.173251.56d94c8c69f/nixos/pkgs/development/interpreters/python/mk-python-derivation.nix:76:3:
while evaluating the attribute 'outPath' at /nix/store/6xrz725n08v8akdh90l8lh7bg3imcz56-nixos-19.03.173251.56d94c8c69f/nixos/lib/customisation.nix:150:7:
while evaluating the attribute 'propagatedBuildInputs' of the derivation 'python3.6-python3.6-bleach-2.1.4' at /nix/store/6xrz725n08v8akdh90l8lh7bg3imcz56-nixos-19.03.173251.56d94c8c69f/nixos/pkgs/development/interpreters/python/mk-python-derivation.nix:76:3:
while evaluating the attribute 'out.outPath' at /nix/store/6xrz725n08v8akdh90l8lh7bg3imcz56-nixos-19.03.173251.56d94c8c69f/nixos/lib/customisation.nix:142:13:
while evaluating the attribute 'propagatedBuildInputs' of the derivation 'python3.6-python3.6-html5lib-1.0.1' at /nix/store/6xrz725n08v8akdh90l8lh7bg3imcz56-nixos-19.03.173251.56d94c8c69f/nixos/pkgs/development/interpreters/python/mk-python-derivation.nix:76:3:
while evaluating the attribute 'out.outPath' at /nix/store/6xrz725n08v8akdh90l8lh7bg3imcz56-nixos-19.03.173251.56d94c8c69f/nixos/lib/customisation.nix:142:13:
while evaluating the attribute 'propagatedBuildInputs' of the derivation 'python3.6-python3.6-lxml-4.3.3' at /nix/store/6xrz725n08v8akdh90l8lh7bg3imcz56-nixos-19.03.173251.56d94c8c69f/nixos/pkgs/development/interpreters/python/mk-python-derivation.nix:76:3:
infinite recursion encountered, at undefined position