Skip to content

Can't lock requirements if platform does not match #5842

@n1ngu

Description

@n1ngu

Issue description

So far we have been listing

[packages]
psycopg2-binary = {version = "*", markers = "platform_machine != 'aarch64'"}
psycopg2 = {version = "*", markers = "platform_machine == 'aarch64'"}

in Pipfiles so that Linux and Mac developers could both benefit from reproducible installs of a project via pipenv.

Expected result

This worked fine, both packages were locked on the same version (most usually) and depending on who issued a pipenv install either one would be actually installed.

Actual result

As of pipenv 2023.08.20 trying to lock on a Linux box will produce a very strange lock file listing psycopg2 on version "*" (what?) with no hashes.

{
    # ...
    "default": {
        # ...
        "psycopg2": {
            "markers": "platform_machine == 'aarch64'",
            "version": "*"
        },
        # ...
}

Output for pipenv lock --verbose:

app@8e1fc420e234:~/asdf$ pipenv lock --verbose
/usr/local/lib/python3.11/site-packages/pipenv/environment.py:20: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pipenv.patched.pip._vendor import pkg_resources
Using python: 3.11
Path to python: /usr/local/bin/python3
Creating a virtualenv for this project...
Pipfile: /app/asdf/Pipfile
Using /usr/local/bin/python3 (3.11.4) to create virtualenv...
⠦ Creating virtual environment...created virtual environment CPython3.11.4.final.0-64 in 346ms
  creator CPython3Posix(dest=/app/asdf/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/app/.local/share/virtualenv)
    added seed packages: pip==23.2.1, setuptools==68.0.0, wheel==0.41.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /app/asdf/.venv
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
/usr/local/lib/python3.11/site-packages/pipenv/environment.py:20: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pipenv.patched.pip._vendor import pkg_resources
Could not find a matching version of psycopg2; platform_machine == "aarch64" for your environment, its dependencies will be skipped.
INFO:pipenv.patched.pip._internal.resolution.resolvelib.factory:Ignoring psycopg2: markers 'platform_machine == "aarch64"' don't match your environment
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.starting()
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('psycopg2-binary'), None)
INFO:pipenv.patched.pip._internal.operations.prepare:Collecting psycopg2-binary (from -r /tmp/pipenv-1gocip9i-requirements/pipenv-aco2fmnp-constraints.txt (line 2))
INFO:pipenv.patched.pip._internal.operations.prepare:Obtaining dependency information for psycopg2-binary from 
https://files.pythonhosted.org/packages/20/81/4940235d18747f865d47eb38b98f38acc24b39278b12e20a0fdd20e0a132/psycopg2_binary-2.9.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
INFO:pipenv.patched.pip._internal.network.download:Using cached psycopg2_binary-2.9.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.4 kB)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.starting_round(0)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.pinning(LinkCandidate('https://files.pythonhosted.org/packages/20/81/4940235d18747f865d47eb38b98f38acc24b39278b12e20a0fdd20e0a132/psycopg2_binary-2.9.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.w
hl (from https://pypi.org/simple/psycopg2-binary/) (requires-python:>=3.6)'))
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.ending_round(0, state)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.starting_round(1)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.ending(State(mapping=OrderedDict([('psycopg2-binary', 
LinkCandidate('https://files.pythonhosted.org/packages/20/81/4940235d18747f865d47eb38b98f38acc24b39278b12e20a0fdd20e0a132/psycopg2_binary-2.9.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (from https://pypi.org/simple/psycopg2-binary/) (requires-python:>=3.6)'))]), 
criteria={'psycopg2-binary': Criterion((SpecifierRequirement('psycopg2-binary'), via=None))}, backtrack_causes=[]))
INFO:pipenv.patched.pip._internal.network.download:Using cached psycopg2_binary-2.9.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
⠹ Locking...
✔ Success!
Warning: Error generating hash for psycopg2.
Locking [dev-packages] dependencies...
Updated Pipfile.lock (fa7e054d3710f4d0124ea9874c1bdf9f42d2c2ff1eb99230f3f503e350116dc8)!

Steps to replicate

# Pifile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
psycopg2-binary = {version = "*", markers = "platform_machine != 'aarch64'"}
psycopg2 = {version = "*", markers = "platform_machine == 'aarch64'"}

[requires]
python_version = "3.11"

Try to run pipenv lock on a Linux box


$ pipenv --support

Pipenv version: '2023.8.20'

Pipenv location: '/usr/local/lib/python3.11/site-packages/pipenv'

Python location: '/usr/local/bin/python'

OS Name: 'posix'

User pip version: '23.2.1'

user Python installations found:

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.11.4',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '6.4.9-1-MANJARO',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP PREEMPT_DYNAMIC Wed Aug  9 08:32:12 UTC 2023',
 'python_full_version': '3.11.4',
 'python_version': '3.11',
 'sys_platform': 'linux'}

System environment variables:

  • PYTHONDEVMODE
  • PIPENV_VENV_IN_PROJECT
  • CELERY_BROKER_URL
  • HOSTNAME
  • PYTHON_VERSION
  • DJANGO_EMAIL_HOST
  • DJANGO_SECRET_KEY
  • PWD
  • DJANGO_SENTRY_ENVIRONMENT
  • DB_ENGINE
  • DJANGO_EMAIL_PORT
  • PYTHON_SETUPTOOLS_VERSION
  • DB_PORT
  • DB_USER
  • HOME
  • LANG
  • DJANGO_SENTRY_DSN
  • GPG_KEY
  • DJANGO_ALLOWED_HOSTS
  • DJANGO_EMAIL_HOST_PASSWORD
  • DJANGO_SETTINGS_MODULE
  • DJANGO_MEDIA_ROOT
  • TERM
  • DB_HOST
  • DB_PASSWD
  • DJANGO_DEBUG
  • SHLVL
  • DJANGO_EMAIL_HOST_USER
  • PYTHON_PIP_VERSION
  • DB_NAME
  • PYTHON_GET_PIP_SHA256
  • PYTHONBREAKPOINT
  • PYTHON_GET_PIP_URL
  • PATH
  • DJANGO_STATIC_ROOT
  • PIPENV_SHELL
  • OLDPWD
  • _
  • PIP_DISABLE_PIP_VERSION_CHECK
  • PYTHONDONTWRITEBYTECODE
  • PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

  • PIPENV_VENV_IN_PROJECT: 1
  • PIPENV_SHELL: bash

Debug–specific environment variables:

  • PATH: /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
  • LANG: C.UTF-8
  • PWD: /app/asdf

Contents of Pipfile ('/app/asdf/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
psycopg2-binary = {version = "*", markers = "platform_machine != 'aarch64'"}
psycopg2 = {version = "*", markers = "platform_machine == 'aarch64'"}

[requires]
python_version = "3.11"

Contents of Pipfile.lock ('/app/asdf/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "fa7e054d3710f4d0124ea9874c1bdf9f42d2c2ff1eb99230f3f503e350116dc8"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.11"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "psycopg2": {
            "markers": "platform_machine == 'aarch64'",
            "version": "*"
        },
        "psycopg2-binary": {
            "hashes": [
                "sha256:00d8db270afb76f48a499f7bb8fa70297e66da67288471ca873db88382850bf4",
                "sha256:024eaeb2a08c9a65cd5f94b31ace1ee3bb3f978cd4d079406aef85169ba01f08",
                "sha256:094af2e77a1976efd4956a031028774b827029729725e136514aae3cdf49b87b",
                "sha256:1011eeb0c51e5b9ea1016f0f45fa23aca63966a4c0afcf0340ccabe85a9f65bd",
                "sha256:11abdbfc6f7f7dea4a524b5f4117369b0d757725798f1593796be6ece20266cb",
                "sha256:122641b7fab18ef76b18860dd0c772290566b6fb30cc08e923ad73d17461dc63",
                "sha256:17cc17a70dfb295a240db7f65b6d8153c3d81efb145d76da1e4a096e9c5c0e63",
                "sha256:18f12632ab516c47c1ac4841a78fddea6508a8284c7cf0f292cb1a523f2e2379",
                "sha256:1b918f64a51ffe19cd2e230b3240ba481330ce1d4b7875ae67305bd1d37b041c",
                "sha256:1c31c2606ac500dbd26381145684d87730a2fac9a62ebcfbaa2b119f8d6c19f4",
                "sha256:26484e913d472ecb6b45937ea55ce29c57c662066d222fb0fbdc1fab457f18c5",
                "sha256:2993ccb2b7e80844d534e55e0f12534c2871952f78e0da33c35e648bf002bbff",
                "sha256:2b04da24cbde33292ad34a40db9832a80ad12de26486ffeda883413c9e1b1d5e",
                "sha256:2dec5a75a3a5d42b120e88e6ed3e3b37b46459202bb8e36cd67591b6e5feebc1",
                "sha256:2df562bb2e4e00ee064779902d721223cfa9f8f58e7e52318c97d139cf7f012d",
                "sha256:3fbb1184c7e9d28d67671992970718c05af5f77fc88e26fd7136613c4ece1f89",
                "sha256:42a62ef0e5abb55bf6ffb050eb2b0fcd767261fa3faf943a4267539168807522",
                "sha256:4ecc15666f16f97709106d87284c136cdc82647e1c3f8392a672616aed3c7151",
                "sha256:4eec5d36dbcfc076caab61a2114c12094c0b7027d57e9e4387b634e8ab36fd44",
                "sha256:4fe13712357d802080cfccbf8c6266a3121dc0e27e2144819029095ccf708372",
                "sha256:51d1b42d44f4ffb93188f9b39e6d1c82aa758fdb8d9de65e1ddfe7a7d250d7ad",
                "sha256:59f7e9109a59dfa31efa022e94a244736ae401526682de504e87bd11ce870c22",
                "sha256:62cb6de84d7767164a87ca97e22e5e0a134856ebcb08f21b621c6125baf61f16",
                "sha256:642df77484b2dcaf87d4237792246d8068653f9e0f5c025e2c692fc56b0dda70",
                "sha256:6822c9c63308d650db201ba22fe6648bd6786ca6d14fdaf273b17e15608d0852",
                "sha256:692df8763b71d42eb8343f54091368f6f6c9cfc56dc391858cdb3c3ef1e3e584",
                "sha256:6d92e139ca388ccfe8c04aacc163756e55ba4c623c6ba13d5d1595ed97523e4b",
                "sha256:7952807f95c8eba6a8ccb14e00bf170bb700cafcec3924d565235dffc7dc4ae8",
                "sha256:7db7b9b701974c96a88997d458b38ccb110eba8f805d4b4f74944aac48639b42",
                "sha256:81d5dd2dd9ab78d31a451e357315f201d976c131ca7d43870a0e8063b6b7a1ec",
                "sha256:8a136c8aaf6615653450817a7abe0fc01e4ea720ae41dfb2823eccae4b9062a3",
                "sha256:8a7968fd20bd550431837656872c19575b687f3f6f98120046228e451e4064df",
                "sha256:8c721ee464e45ecf609ff8c0a555018764974114f671815a0a7152aedb9f3343",
                "sha256:8f309b77a7c716e6ed9891b9b42953c3ff7d533dc548c1e33fddc73d2f5e21f9",
                "sha256:8f94cb12150d57ea433e3e02aabd072205648e86f1d5a0a692d60242f7809b15",
                "sha256:95a7a747bdc3b010bb6a980f053233e7610276d55f3ca506afff4ad7749ab58a",
                "sha256:9b0c2b466b2f4d89ccc33784c4ebb1627989bd84a39b79092e560e937a11d4ac",
                "sha256:9dcfd5d37e027ec393a303cc0a216be564b96c80ba532f3d1e0d2b5e5e4b1e6e",
                "sha256:a5ee89587696d808c9a00876065d725d4ae606f5f7853b961cdbc348b0f7c9a1",
                "sha256:a6a8b575ac45af1eaccbbcdcf710ab984fd50af048fe130672377f78aaff6fc1",
                "sha256:ac83ab05e25354dad798401babaa6daa9577462136ba215694865394840e31f8",
                "sha256:ad26d4eeaa0d722b25814cce97335ecf1b707630258f14ac4d2ed3d1d8415265",
                "sha256:ad5ec10b53cbb57e9a2e77b67e4e4368df56b54d6b00cc86398578f1c635f329",
                "sha256:c82986635a16fb1fa15cd5436035c88bc65c3d5ced1cfaac7f357ee9e9deddd4",
                "sha256:ced63c054bdaf0298f62681d5dcae3afe60cbae332390bfb1acf0e23dcd25fc8",
                "sha256:d0b16e5bb0ab78583f0ed7ab16378a0f8a89a27256bb5560402749dbe8a164d7",
                "sha256:dbbc3c5d15ed76b0d9db7753c0db40899136ecfe97d50cbde918f630c5eb857a",
                "sha256:ded8e15f7550db9e75c60b3d9fcbc7737fea258a0f10032cdb7edc26c2a671fd",
                "sha256:e02bc4f2966475a7393bd0f098e1165d470d3fa816264054359ed4f10f6914ea",
                "sha256:e5666632ba2b0d9757b38fc17337d84bdf932d38563c5234f5f8c54fd01349c9",
                "sha256:ea5f8ee87f1eddc818fc04649d952c526db4426d26bab16efbe5a0c52b27d6ab",
                "sha256:eb1c0e682138f9067a58fc3c9a9bf1c83d8e08cfbee380d858e63196466d5c86",
                "sha256:eb3b8d55924a6058a26db69fb1d3e7e32695ff8b491835ba9f479537e14dcf9f",
                "sha256:ee919b676da28f78f91b464fb3e12238bd7474483352a59c8a16c39dfc59f0c5",
                "sha256:f02f4a72cc3ab2565c6d9720f0343cb840fb2dc01a2e9ecb8bc58ccf95dc5c06",
                "sha256:f4f37bbc6588d402980ffbd1f3338c871368fb4b1cfa091debe13c68bb3852b3",
                "sha256:f8651cf1f144f9ee0fa7d1a1df61a9184ab72962531ca99f077bbdcba3947c58",
                "sha256:f955aa50d7d5220fcb6e38f69ea126eafecd812d96aeed5d5f3597f33fad43bb",
                "sha256:fc10da7e7df3380426521e8c1ed975d22df678639da2ed0ec3244c3dc2ab54c8",
                "sha256:fdca0511458d26cf39b827a663d7d87db6f32b93efc22442a742035728603d5f"
            ],
            "markers": "platform_machine != 'aarch64'",
            "version": "==2.9.7"
        }
    },
    "develop": {}
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions