python subprocess invocation to inject setuptools pollutes setuptools argument list

[marcusva]

setuptools (and probably distutils) assume the very first argument from sys.argv to be the executed setup file. Invoking a python subprocess with -c to execute a command will set '-c' to be the very first argument to sys.argv (see the docs). This leads to the manifest_maker warning noted at

pip/pip/req/req_install.py

Line 868 in 9b11ed2

r"^warning: manifest_maker: standard file '-c' not found$"]:

and also will cause arguments to the setuptools (or distutils) commands to be polluted with an additional '-c', whether it's wanted or not.

As a workaround, manipulate the argument list prior to exec() in

pip/pip/req/req_install.py

Line 843 in 9b11ed2

"import setuptools, tokenize; __file__=%r; exec(compile("

and similar invocations within pip:

import sys, setuptools, tokenize
sys.argv[0] = %r; ....

[koobs]

For more details, see our review and diff here: https://phabric.freebsd.org/D270

[dstufft]

The diff above is no longer available, however this should be pretty easy to fix. It should just require modifying the shim code so that it munges sys.argv to remove the -c and such from it.

[kynan]

@bertilhatt are you working on this?

[kynan]

@pradyunsg Please mark as in progress

[kynan]

@bertilhatt you need to claim this issue on http://gg.gg/PyPASprint2018

[bertilhatt]

Sorry @kynan, the link to gg.gg doesn’t seem to work for me.

I am not working on it anymore.

The pull request is marked as LGTM, so I expect it to go through.

[kynan]

Odd, the short link appears to have expired. https://hackmd.io/JugBUcU6QdCR8QK9fTBfug

[koobs]

Happy 5 year birthday for this issue (last month)! Thank you for resolving this @cjerdonek :)

FWIW, the review/diff URL mentioned in my earlier comment is now https://reviews.freebsd.org/D270

[cjerdonek]

@koobs, per your comment on that thread, would a change to setuptools make for a cleaner fix? Is there a corresponding issue in setuptools' tracker?

[koobs]

@cjerdonek I think the best way to answer that question is wherever the issue is found/reproducible, which at the time of the report, was distutils, setuptools, pip.

I looked through @marcusva's history around the time this issue was created and couldn't see a referenced setuptools bug report