Crash when running a script via an association

[pfmoore]

Describe the bug
Running a Python script with the new manager installed crashes:

❯ xxx
********************************************************************************
To see all available commands, run 'py help'
********************************************************************************
[ERROR] INTERNAL ERROR: LookupError: C:\Users\Gustav\.local\scripts\xxx.py
[ERROR] Internal error 0x00000001. Please report to https://github.com/python/pymanager

To Reproduce
Steps to reproduce the behavior:

1. I have Python 3.13 installed from the old MSI installer, including the py launcher.
2. I have .py in my PATHEXT variable, so running a Python file from the command line works.
3. Typing xxx runs xxx.py, via the shebang line.
4. Install the pymanager application. Do not run on first install, and do not install any Python runtimes.
5. Type xxx at the command line again. This now causes the crash shown above.

This is reproducible - I can uninstall the manager to get the old behaviour, and reinstall it to get the crash.

Expected behavior
The command xxx runs xxx.py, exactly as under the old py launcher.

Additional context
I don't have file associations for .py files. They appear to have been removed at some point in the various modifications I made to try out the new manager previously. I did "associate" .py files wih Python via the explorer GUI, but I don't know where that information is recorded these days (it's clearly not in the registry, see the output of ftype and assoc below).

❯ cmd /c ftype Python.File
File type 'Python.File' not found or no open command associated with it.
❯ cmd /c assoc .py
File association not found for extension .py

Log file:

>> Writing logs to C:\Users\Gustav\AppData\Local\Temp\python_install_20250511225707_34284.log
>> Python installation manager 25.0b5
Copyright (c) Python Software Foundation. All Rights Reserved.

BEGIN install_command.execute: []
>  ********************************************************************************
>  To see all available commands, run 'py help'
>  ********************************************************************************
END install_command.execute
!! INTERNAL ERROR: LookupError: C:\Users\Gustav\.local\scripts\xxx.py
TRACEBACK:
Traceback (most recent call last):
  File "manage\__init__.py", line 50, in main
  File "manage\commands.py", line 806, in execute
  File "manage\install_command.py", line 628, in execute
  File "manage\scriptutils.py", line 168, in find_install_from_script
  File "manage\scriptutils.py", line 163, in _read_script
LookupError: C:\Users\Gustav\.local\scripts\xxx.py

Note that the script itself has no shebang line. It's a 5-line Python script with no dependencies apart from the stdlib.

[pfmoore]

Looking at the code indicated in the log, it appears that the new launcher isn't capable of handling a script without a shebang line 🙁

[zooba]

it appears that the new launcher isn't capable of handling a script without a shebang line

It's meant to raise and then handle the error. It's just the path where it tries to auto-install the runtime you need (because you don't currently have any) isn't handling that case - the section of install_command.py is missing the handler and fallback to cmd.default_tag (the check for automatic installs being disabled has already happened by that point).

[pfmoore]

OK, so what I need is for it to use my installed Python 3.13. Is it only willing to use managed installs? Is this another consequence of the fact that it can't handle the situation where the latest (i.e. default) available version isn't a managed install?

If so, then I could sort of accept that this is a consequence of trying to use this while 3.14 is still in beta. Although (1) that's going to make it hard to get as much testing as you'd like and (2) conceptually I don't think it's reasonable to make managed installs special in this way.

For comparison, uv does something similar with its "managed" vs "system" Python installations, and that's a PITA as well. I don't think we want to end up in a situation where there are three different classes of Python installation that people might have installed simultaneously...

[zooba]

This one is separate from "only willing to use managed installs", though that is a known issue with shebang handling (#91). It's probably two issues here, the crash when it fails to install, and the lack of fallback to an unmanaged runtime when there's no shebang (which is how we ended up in the auto-install path to begin with).

conceptually I don't think it's reasonable to make managed installs special in this way.

Perhaps true, but existing installs simply don't have the metadata we need to make consistent decisions. We can infer some of it for our own runtimes, but I also don't want to give "PythonCore" an unfair advantage (a fair advantage, on the other hand...). There's going to be two tiers regardless, and we can only create a third, we can't remove any.

[pfmoore]

I'm rather uncomfortable about the auto-install behaviour. Can you point me at the documentation of that feature, so that I don't make comments based on a misunderstanding of what's going on?

My initial (under-informed) thoughts, though are:

1. In normal usage¹, the manager is extremely unlikely to be installed without at least one managed install being present. I'd expect that would just get used, no need for auto-install functionality.
2. I can see auto-installing based on script metadata stating a python-requires - it's the same idea as auto-installing dependencies.
3. I'd be more surprised if #!/usr/bin/python3.12 did an auto-install. My expectation there would be an error message ("Python 3.12 needed but not available") that the user could choose to fix with a manual install.

In particular, if I have Python 3.14 installed as my default Python, I would not want running a script that had a #!python3.15 shebang to silently change my default Python version². Actually, that would be true for case (2) as well - script metadata is typically used to set up an isolated environment for the script, so causing a global change is bad in that case as well.

Footnotes

1. i.e. not my weird testing environment 🙂 ↩

2. I might not even notice for a while, which could spread the breakage into new virtual environments. ↩

[zooba]

1. n normal usage1, the manager is extremely unlikely to be installed without at least one managed install being present. I'd expect that would just get used, no need for auto-install functionality

Correct, scripts without a shebang should use any existing install (including an unmanaged one), and so it's extremely unlikely for an unintended install to happen automatically (unless there are no installs at all).

2. I can see auto-installing based on script metadata stating a python-requires - it's the same idea as auto-installing dependencies.

The functionality for this is kinda there, but searching for the script metadata is currently disabled (or not implemented - I implemented it once but can't remember if I left in it there or not).

3. I'd be more surprised if #!/usr/bin/python3.12 did an auto-install

Prepare to be surprised, I guess 😄 That one should definitely do an install for you, based on the first install in the default feed that offers the python3.12 alias. There's config/envvars to disable auto install, but I'd rather people be surprised during beta to find out if it's pleasant or unpleasant.

There is a documentation gap, though. It would be in the shebangs section, but currently isn't. python/cpython#133928

[zooba]

In particular, if I have Python 3.14 installed as my default Python, I would not want running a script that had a #!python3.15 shebang to silently change my default Python version

Yeah, this has the potential to be surprising. I expect once people are surprised once then they'll set the configuration to fix their default tag and forget about it (and then complain that they don't have 3.15 when they want it), but maybe the fact that the install isn't silent will be enough?

[pfmoore]

Yeah, this has the potential to be surprising.

Sorry, I'm going to say this is way worse than "surprising". It'll be rare, probably very rare, but it'll be nasty for anyone it hits.

maybe the fact that the install isn't silent will be enough?

Problem is, seeing it isn't enough, you have to know what to do to fix the problem (in fact, you have to know enough to realise that it is a problem!) If it were me, I'd probably spot it and frantically start hitting Ctrl-C to make it stop, causing more of a mess than just letting it run would have done 🙁 Could we check, and if the installation would change the default version, stop and prompt - "Running this script will require setting your default Python version to 3.15 - are you sure? [y/N]" (note, the default is "no").

I expect once people are surprised once then they'll set the configuration to fix their default tag and forget about it

Hey - are there two Steves, one posting here and one on Discourse? The guy on Discourse is going to hate that last comment 🙂

[zooba]

Could we check, and if the installation would change the default version, stop and prompt - "Running this script will require setting your default Python version to 3.15 - are you sure? [y/N]

I actually don't mind prompting unconditionally before an automatic install here (well, conditional on the "confirm" setting). I do like the idea of automatic deployments being able to just run the script they rolled out without thinking about whether Python has been installed or not, but the impact to interactive users is probably surprising enough to justify the warning.

Hey - are there two Steves, one posting here and one on Discourse?

I have draft posts open everywhere right now 🤦‍♂ Not sure if the context switching would be better or worse if we were sitting in the same office!