Cross-platform Pipenv.lock?

[AlexP11223]

I am creating a cross-platform Python app.

While trying to build the app on Windows and Linux I noticed that Pipenv.lock is platform-specific.

For example, I create a project on Windows and add pytest, it has colorama with "markers": "sys_platform == 'win32'" in the lock file. But when modifying the lock file on Linux, such as adding a new lib (pipenv install hello-world), this dependency will disappear.

diff ../py-win/Pipfile.lock Pipfile.lock 
4c4
<             "sha256": "828b8ad012f4c8773e6e61e3ac2be0ffcd7540fd7ed175a8355676c8e31c4d3d"
---
>             "sha256": "c4a1bdfad8e2d208344003d9c704432100fbcb9bd87f713178895637222767a4"
33c33
<         "colorama": {
---
>         "hello-world": {
35,36c35
<                 "sha256:05eed71e2e327246ad6b38c540c4a3117230b19679b875190486ddd2d721422d",
<                 "sha256:f8ac84de7840f5b9c4e3347b3c1eaa50f7e49c2b07596221daec5edaabbd7c48"
---
>                 "sha256:f236ab36224613dff8db190dc4a02b3787c3d19e95f7e0876e3098966118bc1e"
38,39c37,38
<             "markers": "sys_platform == 'win32'",
<             "version": "==0.4.1"
---
>             "index": "pypi",
>             "version": "==0.2"

Here is the full output on both systems: https://gist.github.com/AlexP11223/6457fd4c98272c526c855c63088c00d4

Is it supposed to work like that? If yes, what is the best way to handle that for cross-platform apps? Is there any better solution than just committing lock files for all platforms to the Git repo (such as Pipenv-win.lock, Pipenv-linux.lock) and renaming one of them to Pipenv.lock when building?

Pipenv version: '2018.11.26'

Pipenv location: 'C:\\Users\\user\\AppData\\Roaming\\Python\\Python37\\site-packages\\pipenv'

Python location: 'c:\\python37\\python.exe'

Python installations found:

• 3.7.3: C:\Python37\python.exe

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.3',
 'os_name': 'nt',
 'platform_machine': 'AMD64',
 'platform_python_implementation': 'CPython',
 'platform_release': '10',
 'platform_system': 'Windows',
 'platform_version': '10.0.17763',
 'python_full_version': '3.7.3',
 'python_version': '3.7',
 'sys_platform': 'win32'}

Pipenv version: '2018.11.26'

Pipenv location: '/home/alex/.local/lib/python3.7/site-packages/pipenv'

Python location: '/usr/bin/python3'

Python installations found:

• 3.7.3: /usr/bin/python3.7
• 3.7.3: /usr/bin/python3.7m
• 2.7.16: /usr/bin/python

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.3',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.0.0-21-generic',
 'platform_system': 'Linux',
 'platform_version': '#22+system76-Ubuntu SMP Tue Jul 16 19:57:52 UTC 2019',
 'python_full_version': '3.7.3',
 'python_version': '3.7',
 'sys_platform': 'linux'}

[frostming]

use --keep-outdated to keep the unneeded dependencies on a platform.

[AlexP11223]

it doesn't seem to work

alex@pop-os:~/py1$ pipenv install --keep-outdated
Creating a virtualenv for this project…
Pipfile: /home/alex/py1/Pipfile
Using /usr/bin/python3.7 (3.7.3) to create virtualenv…
⠋ Creating virtual environment...

⠙ Creating virtual environment...

⠴ Creating virtual environment...Using base prefix '/usr'
New python executable in /home/alex/.local/share/virtualenvs/py1-MRCLG4IM/bin/python3.7
Also creating executable in /home/alex/.local/share/virtualenvs/py1-MRCLG4IM/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python3.7

✔ Successfully created virtual environment! 
Virtualenv location: /home/alex/.local/share/virtualenvs/py1-MRCLG4IM
Installing dependencies from Pipfile.lock (1c4d3d)…
Ignoring colorama: markers 'sys_platform == "win32"' don't match your environment
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 13/13 — 00:00:01
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
alex@pop-os:~/py1$ pipenv install --keep-outdated hello-world
Installing hello-world…
Adding hello-world to Pipfile's [packages]…
✔ Installation Succeeded 
Pipfile.lock (2767a4) out of date, updating to (1c4d3d)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✔ Success! 
Updated Pipfile.lock (2767a4)!
Installing dependencies from Pipfile.lock (2767a4)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 13/13 — 00:00:01
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

alex@pop-os:~/py1$ diff ../py-win/Pipfile.lock Pipfile.lock
4c4
<             "sha256": "828b8ad012f4c8773e6e61e3ac2be0ffcd7540fd7ed175a8355676c8e31c4d3d"
---
>             "sha256": "c4a1bdfad8e2d208344003d9c704432100fbcb9bd87f713178895637222767a4"
33c33
<         "colorama": {
---
>         "hello-world": {
35,36c35
<                 "sha256:05eed71e2e327246ad6b38c540c4a3117230b19679b875190486ddd2d721422d",
<                 "sha256:f8ac84de7840f5b9c4e3347b3c1eaa50f7e49c2b07596221daec5edaabbd7c48"
---
>                 "sha256:f236ab36224613dff8db190dc4a02b3787c3d19e95f7e0876e3098966118bc1e"
38,39c37,38
<             "markers": "sys_platform == 'win32'",
<             "version": "==0.4.1"
---
>             "index": "pypi",
>             "version": "==0.2"

[frostming]

Would you like to try the master branch?

[AlexP11223]

What's the simplest way to install it?

[frostming]

@AlexP11223 If you are on mac: brew install pipenv --HEAD
Otherwise: pip install --user -e git+https://github.com/pypa/pipenv.git@master

[AlexP11223]

Looks like it works in master:

alex@pop-os:~/py2$ python3 -m pipenv install --keep-outdated
Creating a virtualenv for this project…
Pipfile: /home/alex/py2/Pipfile
Using /usr/bin/python3.7m (3.7.3) to create virtualenv…
⠙ Creating virtual environment...Using base prefix '/usr'
New python executable in /home/alex/.local/share/virtualenvs/py2-5HLUPQ8M/bin/python3.7m
Also creating executable in /home/alex/.local/share/virtualenvs/py2-5HLUPQ8M/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python3.7m

✔ Successfully created virtual environment! 
Virtualenv location: /home/alex/.local/share/virtualenvs/py2-5HLUPQ8M
Installing dependencies from Pipfile.lock (1c4d3d)…
Ignoring colorama: markers 'sys_platform == "win32"' don't match your environment
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 13/13 — 00:00:11
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
alex@pop-os:~/py2$ python3 -m pipenv install --keep-outdated hello-world
Installing hello-world…
Adding hello-world to Pipfile's [packages]…
✔ Installation Succeeded 
Pipfile.lock (1c4d3d) out of date, updating to (2767a4)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Building requirements...
Resolving dependencies...
✔ Success! 
Updated Pipfile.lock (2767a4)!
Installing dependencies from Pipfile.lock (2767a4)…
Ignoring colorama: markers 'sys_platform == "win32"' don't match your environment
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 14/14 — 00:00:01
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
alex@pop-os:~/py2$ python3 -m pipenv --version
pipenv, version 2018.11.27.dev0

alex@pop-os:~/py2$ diff ../py-win/Pipfile.lock Pipfile.lock
4c4
<             "sha256": "828b8ad012f4c8773e6e61e3ac2be0ffcd7540fd7ed175a8355676c8e31c4d3d"
---
>             "sha256": "c4a1bdfad8e2d208344003d9c704432100fbcb9bd87f713178895637222767a4"
39a40,46
>         },
>         "hello-world": {
>             "hashes": [
>                 "sha256:f236ab36224613dff8db190dc4a02b3787c3d19e95f7e0876e3098966118bc1e"
>             ],
>             "index": "pypi",
>             "version": "==0.2"

[AlexP11223]

btw looks like Poetry handles it better by default, it doesn't remove dependencies for other platforms https://gist.github.com/AlexP11223/afbe247175efdcc897930d8c6ab04184

Maybe pipenv should consider this too?

[frostming]

@AlexP11223 --keep-outdated does the trick, as said above

[AlexP11223]

I mean that it should be the default behavior (otherwise someone will forget to add it sooner or later), not hidden under some switch with an unrelated name (which probably does more than just keeping dependencies marked as for other platforms).