Remove shebang and executable bit from most modules

[jaraco]

Feature or enhancement

Proposal:

In jaraco/backports.tarfile#4, I discovered that the presence of a shebang and executable bit on the tarfile module signals to downstream packagers and users that the module can be treated like a script and executed directly. That expectation seems to stem from an incidental artifact of the original contribution happening to have been written as a script. In fact, back in those days, many developers including myself would add #!/usr/bin/env python to a Python file just to signal that the syntax was Python, so it may never have been intended to be executed directly.

Nevertheless, the presence of the shebang led to it being marked as executable in #64135.

Marking this module as executable prevents the module from ever being ported to a package (such as backports.tarfile does) without breaking that assumption. It also tempts users to execute a module with the incorrect Python version. Instead, users should be directed to runpy invocation (i.e. python -m tarfile).

This proposal only includes tarfile in the scope, but it also implies that other modules changed in #64135 should also have their shebangs removed except where explicitly desired (and presumably documented or tested as such).

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• gh-118673: Remove shebang and executable bits from tarfile. #118674
• gh-118673: Remove shebang and executable bits from stdlib modules. #119658

[jaraco]

In the prior bug, gpshead indicated that these shebangs should probably be removed.

@serhiy-storchaka Do you have any objections to removing executable bits and shebang from tarfile and maybe others?

[serhiy-storchaka]

I think it made more sense in the past, when:

• python -m did not exist.
• There was usually only one system Python.
• Python module (like pydoc) could be turned into a command by a symlink.
• In general, the stdlib code supported compatibility with several Python versions.

Today this is perhaps no longer relevant or less relevant. But it needs to be discussed with a wider audience to shape general policy.

[jaraco]

Summarized and posted in discourse.

[merwok]

Previously reported in #85792 (comment) :

For stdlib modules, I think the intended usage is python3 -m encodings.rot_13; I don’t expect anyone to run python3 /usr/lib/python3.10/encodings/rot_13.py to do a decoding in the terminal!
So for these easter eggs / hidden gems in the stdlib, I would delete the shebang and remove the executable bit. There are 18 such modules, ignoring test, lib2to3, turtledemo and a few others named below.

There are 15 modules left now (+ 14 in turtledemo); I think there should be 1 PR rather than 15, and it would not have to be backported (let’s avoid the work + debating the definition of bugfix + xkcd 1172)

[jaraco]

Sounds good. The consensus in the discourse is that we can remove them. I'll start a new PR as such.

[jaraco]

I note that there are only 14 modules in Lib, now that cgi.py is gone.

[XuehaiPan]

How about adding pre-commit checks for this?

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.6.0
    hooks:
      - id: check-executables-have-shebangs
      - id: check-shebang-scripts-are-executable
      - id: check-symlinks
      - id: destroyed-symlinks

---

cpython/.pre-commit-config.yaml

Lines 22 to 34 in 94e9585

	- repo: https://github.com/pre-commit/pre-commit-hooks
	rev: v4.5.0
	hooks:
	- id: check-case-conflict
	- id: check-merge-conflict
	- id: check-toml
	exclude: ^Lib/test/test_tomllib/
	- id: check-yaml
	- id: end-of-file-fixer
	types: [python]
	exclude: Lib/test/tokenizedata/coding20731.py
	- id: trailing-whitespace
	types_or: [c, inc, python, rst]

[jaraco]

That seems reasonable. Probably relates more to the earlier PR to make things consistent than to this one (as the checks would pass both before and after). Feel free to prepare a PR.

[terryjreedy]

Not backporting this to 3.13 and 3.12 at least for turtledemo examples made it impossible to backport fixes to their docstrings without getting a merge conflict. They all (most?) start with something like the following, with an unneeded first line and a now incorrect file name.

"""     turtle-example-suite:

             tdemo_bytedesign.py

I considered asking about backporting just the turtledemo changes, but it might be easier to write a program to make the changes and apply it to both main and 3.13 branches.