PEP 394: Allow the python command to not be installed, and other minor edits
#630
Conversation
Relax recommendations on the Unix ``python`` command (which should invoke Python 2) in these cases: - Users and administrators can, by a deliberate action, change ``python`` to invoke Python 3. (Activating a venv counts as such an action.) - Distributions can omit the ``python`` command even when ``python2`` is installed in test environments, build systems, and other controlled environments where being explicit is valued more than user experience.
Did you mean I'm also still unhappy with any kind of endorsement of Regardless of what macOS does I think I would be happier in a future where |
And are you more happy with the current state where
Given python2 dies in less than two years, I am afraid that future of yourse will be really really short ;). (still not yet speaking as the SUSE Python maintainer, but I expect to get there on May 2nd ;)) |
|
The endorsement of "python" -> "python2" exists regardless of what you put in a PEP. You are hopelessly naive expecting that use of Python 2 will die at the time support is stopped. In fact IIRC Red Hat was at some point expecting to make money supporting it past that deadline (not sure if that's still their expectation). |
|
On 04/25/18 16:03, Guido van Rossum wrote:
The |python| command is still not available out-of-the box on macOS,
so it didn't completely live up to the expectation of being the
cross-platform way to launch 2/3 source compatile scripts.
Did you mean |python2| there? In my experience macOS comes with |python|
installed (and invoking Python 2) but no |python2| link (hard or soft).
In any case I'm not sure how this strengthens your argument.
Indeed, that's a thinko, sorry.
The python2 command is still not available out-of-the box on macOS, so
it didn't completely live up to the expectation of being the
cross-platform way to launch python2 scripts.
I'm also still unhappy with /any/ kind of endorsement of |python|
pointing to |python3|. When a user gets bitten by this they should
receive an apology from whoever changed that link, not a haughty "the
PEP endorses this".
That's why I'm suggesting this should *only* be a deliberate change
initiated by the user or sysadmin, and that its documentation should
point to the PEP.
People are already doing this, mostly in a (from a packager's
perspective) horrible way -- changing a file installed by a package manager.
I want to allow them do it in a controlled way, which includes a pointer
to the PEP.
Regardless of what macOS does I think I would be happier in a future
where |python| doesn't exist and one always has to specify |python2| or
|python3|. Quite possibly there will be an age where Python 2, 3 and 4
all overlap, and EIBTI.
I'm afraid that's a losing battle.
Why not start with venv (and watch the backlash)?
I wonder how we'll convince macOS to go there.
Anyway, this is my reason for the other proposed change -- I want to
ensure nothing packaged for Fedora should use /usr/bin/python.
|
No, I'm not expecting that. |
|
FWIW Homebrew does include a python2 -> python symlink, so e.g. if you're installing 2.7.14 from brew, |
|
Yes, Homebrew found out the hard way that PEP 394 is actually relatively well thought-out and should be followed. |
|
Forget about whatever Apple does - that shouldn't influence our decision here. Apple's out of the box distribution of open source tools isn't a model of modernity or utility. It's only whatever works for them. Homebrew (what I use) and maybe Fink or MacPorts (distributions I no longer use) are probably more receptive to whatever we decide. I know for a fact that Debian/Ubuntu won't do anything about Having said all that, I think it does make sense to allow distributions to make PEP-aligned policy as they see fit. They know their communities, policies, implications, etc. better than "we" do. OTOH, I'm also happy waiting until 2.7 EOL to make any changes to this PEP's policy. (Maybe we can say, here's where the PEP stands post-2020). |
|
To be clear, what I would endorse is a world where I haven't memorized PEP 394 -- if it currently stipulates that (I do think venv is wrong too, but it's too late, and perhaps it's the compromise we're all looking for.) |
|
On 04/25/18 16:56, Guido van Rossum wrote:
To be clear, what I /would/ endorse is a world where |python| simply
doesn't exist for most users. That seems compatible with what Fedora is
striving for. I don't have any contacts inside Apple any more, so we'll
have to exclude macOS.
OK. That's the main change I want to get through.
I can limit the PR to just that.
I haven't memorized PEP 394 -- if it currently stipulates that |python|
/must/ exist we should definitely drop that requirement.
Yes, the PEP currently says "The more general `python` command should be
installed whenever any version of Python 2 is installed and should
invoke the same version of Python as the `python2` command".
This is the main thing I find restrictive.
But we
shouldn't replace it with any kind of endorsement of people making
|python| point to |python3|.
The PEP also says things like:
"If [...] sysadmins are permitted to change the python command, then the
python command should always be implemented as a link to the interpreter
binary (or a link to a link) and not vice versa. [...]"
IOW, users changing python to python3 is already there; this change is
making it more visible, explaining how to do it properly, and aligning
it with venvs.
Should this rather be removed entirely?
|
|
Also, the PEP currently explicitly says:
|
I will let others argue over this, they seem to be smarter than me, but just let me note here that so far I am an employee of Red Hat, so I know something about the support of EOL'ed programs. |
| similar tool) is active, the ``python`` command should refer to the | ||
| virtual environment's interpreter. In other words, activating a virtual | ||
| environment counts as deliberate user action to change the default | ||
| ``python`` interpreter. |
There was a problem hiding this comment.
+1 on this bullet point. Having python point to Python 3 when a Python 3 environment is activated is now established practice, and it would be nice for the PEP to acknowledge that. Many subparts of the community have already shifted away from the old idiom of relying on the system Python for user scripts.
There was a problem hiding this comment.
Yeah, I won't ask venv to change, I just rm <env>/bin/python after creating the env. :-) I wonder if it should say "may refer" rather than "should refer"?
I do note that this makes using #!/usr/bin/env python quite unreliable to invoke Python 2.
There was a problem hiding this comment.
I'd rather keep "should". It's quite an engrained expectation by now to have python point to the environment's Python. Doing otherwise would break those expectations.
There was a problem hiding this comment.
Aye, this is definitely a should - we rely on it heavily in the Python Packaging User Guide.
pep-0394.txt
Outdated
| refers to the same target as ``python2``. | ||
| * for the time being, all distributions *should* ensure that ``python``, | ||
| if installed, refers to the same target as ``python2``, unless the system | ||
| administrator or user deliberately override this. |
There was a problem hiding this comment.
So this should probably change to ensuring that python, if it exists, runs Python 2 unless (a) the user explicitly overrides it, or (b) a venv is active.
pep-0394.txt
Outdated
| Python as the ``python2`` command (however, note that some distributions | ||
| have already chosen to have ``python`` implement the ``python3`` | ||
| command; see the `Rationale`_ and `Migration Notes`_ below). | ||
| command, and system administrators may be allowed to change the default; |
There was a problem hiding this comment.
I don't want sysadmins to feel empowered by this PEP, unless they are the only user of a system. Sysadmins often don't know what their users are doing. Changing what python does should be up to the user. The sysadmin should not have any dependencies on python -- sysadmin scripts should use python2 or python3.
pep-0394.txt
Outdated
| command, and system administrators may be allowed to change the default; | ||
| see further recommendations and the `Rationale`_ and `Migration Notes`_ | ||
| below). | ||
| * The ``python`` command should be available whenever ``python2`` is available, |
pep-0394.txt
Outdated
| * When packaging software that is source compatible with both versions, | ||
| distributions may change such ``python`` shebangs to ``python3`` (or | ||
| ``python2``). This ensures software is used with the latest version of | ||
| Python available, and it can remove a dependency on Python 2. |
There was a problem hiding this comment.
(Then why the "or python2" ?)
pep-0394.txt
Outdated
| (All software in such a controlled environment must use ``python3`` or | ||
| ``python2`` rather than ``python``, which means scripts that deliberately | ||
| use ``python`` need to be modified for such environments.) | ||
| * System administrators and users may be empowered to change the meaning of |
pep-0394.txt
Outdated
| in March and July 2011 ([1]_, [2]_), February 2012 ([4]_) and | ||
| September 2014 ([6]_). | ||
| in March and July 2011 ([1]_, [2]_), February 2012 ([4]_), | ||
| September 2014 ([6]_), and April 2018 (XXX). |
There was a problem hiding this comment.
Likely this discussion isn't going to be on python-dev but just in this PR.
pep-0394.txt
Outdated
| refer to ``python3`` rather than ``python2``. | ||
| party ecosystem surrounding Python 2 becomes irrelevant in most use cases, | ||
| at which point this recommendation should be updated to suggest that the | ||
| ``python`` symlink refer to ``python3`` rather than ``python2``. |
There was a problem hiding this comment.
Honestly I would rather drop this paragraph -- it spreads false hope.
pep-0394.txt
Outdated
| @@ -150,15 +175,15 @@ making such a change. | |||
| * When the ``pythonX.X`` binaries are provided by a distribution, the | |||
| ``python2`` and ``python3`` commands should refer to one of those files | |||
| rather than being provided as a separate binary file. | |||
| * It is suggested that even distribution-specific packages follow the | |||
| ``python2``/``python3`` convention, even in code that is not intended to | |||
| * It is suggested that distribution-specific packages use ``python2`` or | |||
There was a problem hiding this comment.
I'd say "strongly encouraged" or something like that.
pep-0394.txt
Outdated
| convention by changing the ``python`` interpreter on a test box and checking | ||
| to see if anything breaks. | ||
| convention by changing or removing the ``python`` command on a test box | ||
| and checking to see if anything breaks. |
There was a problem hiding this comment.
Perhaps this fairly lame testing recommendation should just be deleted.
pep-0394.txt
Outdated
| has ``python`` in its shebang line, is invoked on a system that does not have | ||
| Python 2 (and thus, the ``python`` command) installed. | ||
| This is mostly a non-issue -- as with any other case of a required interpreter | ||
| not being installed, the sysadmin can install the ``python`` command. |
There was a problem hiding this comment.
Again, I don't want sysadmins to feel empowered by this. Assuming the shebang line uses #!/bin/env python this can be remedied by activating a venv.
As is the case for the Python 2.7.x's provided by python.org macOS installers. And the MacPorts Python 2.7.x. Unfortunately, Apple support of Python in macOS releases has languished (for example, no Python 3, not up-to-date 2.7, no pip, no python2 link, seriously buggy outdated Tk, deficient SSL support) to the point that it is generally a disservice to users to recommend using it. So I don't think the Apple-supplied system Python should be a factor in discussions here, other than the fact that it does provide /usr/bin/python and /usr/bin/python2.7 links that most-third party macOS Python distributors shadow by $PATH manipulation (and will continue to need to do so as least as long as Apple keeps supplying them in /usr/bin). |
There was a problem hiding this comment.
I'm personally fine with this weakened version. I wonder if we should post a summary of the changes to python-dev to put people on notice that we're changing the PEP. (IMO the most serious change is allowing python to not exist at all, and the second-most serious change is the removal of the paragraph about the anticipated future where python points to Python 3 -- the rest I see as minor edits.)
| similar tool) is active, the ``python`` command should refer to the | ||
| virtual environment's interpreter. In other words, activating a virtual | ||
| environment counts as deliberate user action to change the default | ||
| ``python`` interpreter. |
There was a problem hiding this comment.
Yeah, I won't ask venv to change, I just rm <env>/bin/python after creating the env. :-) I wonder if it should say "may refer" rather than "should refer"?
I do note that this makes using #!/usr/bin/env python quite unreliable to invoke Python 2.
pep-0394.txt
Outdated
| @@ -267,6 +276,9 @@ References | |||
| .. [6] PEP 394 - Clarification of what "python" command should invoke | |||
| (https://mail.python.org/pipermail/python-dev/2014-September/136374.html) | |||
|
|
|||
| .. [7] PEP 394: Allow changing the `python` command in some cases | |||
There was a problem hiding this comment.
I suggest changing the PR title to match the contents.
|
Maybe the python command should just print "Did you mean python2 or
python3?" to stderr and fail?
Steve Holden
…On Fri, Apr 27, 2018 at 9:10 AM, Antoine Pitrou ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In pep-0394.txt
<#630 (comment)>:
> + is valued over user experience (for example, in test environments and
+ package build systems), distributions may choose to not provide the
+ ``python`` command even if ``python2`` is available.
+ (All software in such a controlled environment must use ``python3`` or
+ ``python2`` rather than ``python``, which means scripts that deliberately
+ use ``python`` need to be modified for such environments.)
+* System administrators and users may be empowered to change the meaning of
+ the ``python`` command to ``python3``. This should be a deliberate action
+ by the administrator. Related documentation should include a discussion
+ of the effects -- for example, a link to the "Migration Notes" section of
+ this PEP.
+* When a virtual environment (created by the PEP 405 ``venv`` package or a
+ similar tool) is active, the ``python`` command should refer to the
+ virtual environment's interpreter. In other words, activating a virtual
+ environment counts as deliberate user action to change the default
+ ``python`` interpreter.
I'd rather keep "should". It's quite an engrained expectation by now to
have python point to the environment's Python. Doing otherwise would
break those expectations.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#630 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAGbYO1x0Ij1pyRmK4LfVjbe7mU1BVwmks5tstJigaJpZM4Tj_Yx>
.
|
encukou
changed the title
python command in some casespython command to not be installed, and other minor edits
That tends to confuse tools like buildsystems which detect presence of the |
|
Good point.
Steve Holden
…On Fri, Apr 27, 2018 at 4:01 PM, Petr Viktorin ***@***.***> wrote:
Maybe the python command should just print "Did you mean python2 or
python3?" to stderr and fail?
That tends to confuse tools like buildsystems which detect presence of the
python command. I'm not saying they *should* be doing that, just that a
failing command is quite different from the command not being available.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#630 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAGbYJkiHd5pVM75ehmCHPWZSdfwdew2ks5tszLEgaJpZM4Tj_Yx>
.
|
|
@encukou Let me know when you feel this is ready to commit. Please remind me to change commit topic line to match what you put in the PR. :-) |
|
This PEP was written a while back, so good to be revisiting. But I also think we need to acknowledge common practice -- I doubt we will ever get to the day when there is no "python" command, and I have no idea what the numbers are, but there are certainly folks out there that are symlinking "python" to "python3" right now -- it's part of the "python 3 IS python now" approach. So maybe the paragraph about the anticipated future where python points to Python 3 shouldn't be removed, but rather edited -- I suspect that will happen in some environments regardless of what this PEP says. |
|
No, I really want that paragraph deleted. My point is that the PEP should not encourage hopes in that direction or endorse actions by rogue sysadmins (or distros :-), other than the established practice in virtual environments. I want people who write 3rd party documentation to be quite aware of the repercussions when they suggest their readers type And I want sysadmins (and even users and distros) who "helpfully" link |
|
@warsaw @kerrickstaley This version looks good to me, and I'm the only one that's commented on @encukou's python-dev thread so far, so unless one of you raises any objections, I'm going to suggest that Petr add himself as a co-author, and then we merge the update. |
|
I don't object to adding Petr as a co-author, but I don't see it as
necessary either if the original authors approve of his suggestions. Author
credits should be reserved for those willing to stand by the document long
term, not handed out to every one-time contributor (those are traditionally
listed under Acknowledgments). It's up to you three as authors how to
handle this.
|
|
Right, the author credit here wouldn't just be about these particular updates - it's also about Petr being one of the primary drivers of how Fedora is handling the migration, so he is one of the folks having to stand by the recommendations it provides. (I agree it can be a separate question, it just occurred to me to ask it while writing my earlier comment) |
encukou
changed the title
python command to not be installed, and other minor editspython command to not be installed, and other minor edits
|
@ncoghlan Sorry, just been too busy to dive deeply into this, but I have lured on thread and this PR. JFDI! (And I'm find with adding @encukou as an author.) The one "future" point I wholeheartedly agree with is that if we ever do a Python 4, I do not want to add |
|
I don't feel that great about taking new long-term responsibilities, but discussions on this PEP are probably something I'll get involved in anyway. Since the other authors don't object, let's make it explicit. |
|
LGTM
…On Sat, Apr 28, 2018 at 8:42 AM, Petr Viktorin ***@***.***> wrote:
I don't feel that great about taking new long-term responsibilities, but
discussions on this PEP are probably something I'll get involved in anyway.
Since the other authors don't object, let's make it explicit.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#630 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ACwrMimVxndZ_IZUL6likHgT5JEh1TT1ks5ttI3PgaJpZM4Tj_Yx>
.
--
--Guido van Rossum (python.org/~guido)
|
|
I'm OK with adding Petr as an author if the rest of you are.
…On Sat, Apr 28, 2018, 2:51 PM Petr Viktorin ***@***.***> wrote:
Merged #630 <#630>.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#630 (comment)>, or mute the
thread
<https://github.com/notifications/unsubscribe-auth/AALRxXlUTU1mgfSET0yXFdz7peojNlmbks5ttORvgaJpZM4Tj_Yx>
.
|
|
Thanks for working through this @encukou, and @gvanrossum for the initial review (I get the impression the original PR looked quite different from the final version I reviewed!) |
The intended future for the ``python`` command is: > ``python`` doesn't exist and one always has to specify ``python2`` > or ``python3``. ( – Guido, python#630 (comment) ) There are three conflicting ideas around the ``python`` command, longer-term: 1. The ``python`` command should continue to refer to Python 2 (if Python 2 is available). 2. ``python`` is a correct shebang for py2/py3 source-compatible scripts. 3. Python 2 should *not* be available (by default / after 2010). One of these has to give. It seems that (2) is the easiest to shed, so this proposal does just that. * Make ``python3`` the preferred shebang for py2/py3 compatible scripts, as that's the only shebang that'll work on py2-less systems. (An exception is made for scripts targetting the *system* Python on e.g. macOS/RHEL.) * Make the unversioned ``python`` command optional (along with ``idle``, ``pydoc``, and ``python-config``). Distributions now do not need to install it by default, even if Python 2 is installed. (But they should make it installable explicitly, as long as they ship Python 2.) This should introduce more people to systems without the "legacy" ``python`` command, encourage the use of explicit ``python2``/``python3``, and ease switching to systems that don't have Python 2 at all. * Clarify that distributions *should not* make unversioned ``python`` configurable. (That might work for carefully managed systems, but the ecosystem should converge on ``python3``, not "your ``python`` needs to be set properly".) * Remove mentions of choosing to link ``python`` to ``python3`` in the future, as we don't expect to start recommending that. * Use the term **"unversioned"** ``python`` when contrasting it with ``python3``/``python2``. I found that this makes the message much clearer.
Please do not merge yet. I'll send this for discussion on python-dev, but I think a concrete pull request makes the review easier.
In Fedora, I found that PEP 394's strict recommendation that
pythonpoints topython2is holding us back. I would like to officially relax this in several cases.The problems are:
pythoninvokespython2sends a strong signal that 2 is somehow the preferred version, and it's OK to start new projects in it.and sysadminsthat do want to “live in the future” are switching the symlink topython3themselves. We would like to give them a supported, documented way to do so -- and make surer they're aware of the caveats.python2command is still not available out-of-the box on macOS, so it didn't completely live up to the expectation of being the cross-platform way to launch python2 scripts.pythonin the shebang line can mean either that a script is carefully written to be 2/3 compatible, or that the author/maintainer is lazy or unaware of the recommendations. While Fedora guidelines have long banned the unversioned command, we feel that the only way to enforce that guidelines is to provide environments where thepythoncommand does not work (unless explicitly installed).To help solve these, I would like to relax recommendations on the Unix
python -> python2symlink in these cases:and administratorscan, by a deliberate action, changepythonto invoke Python 3. (Activating a venv counts as such an action, but so would e.g. using alternates, installing a non-default overriding package, or replacing /usr/bin/python.)pythoncommand even whenpython2is installed.This PR also spells out several other things, which I felt were hidden between the lines -- but correct me if you disagree with my reading.