Skip to content

asyncio blocks custom BaseException exceptions #108711

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 tasks done
CendioOssman opened this issue Aug 31, 2023 · 3 comments
Closed
2 tasks done

asyncio blocks custom BaseException exceptions #108711

CendioOssman opened this issue Aug 31, 2023 · 3 comments
Labels
topic-asyncio type-bug An unexpected behavior, bug, or error

Comments

@CendioOssman
Copy link
Contributor

CendioOssman commented Aug 31, 2023

Bug report

Checklist

  • I am confident this is a bug in CPython, not a bug in a third-party project
  • I have searched the CPython issue tracker,
    and am confident this bug has not been reported before

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.11.4 (main, Jun 7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)]

A clear and concise description of the bug:

Since bpo-32528, this code no longer works:

import asyncio

class ShowStopper(BaseException):
    pass

def hello_world(loop):
    print('Hello World')
    raise ShowStopper("Stop")

loop = asyncio.get_event_loop()

loop.call_soon(hello_world, loop)

loop.run_forever()
loop.close()

I think this behaviour is very surprising as the general behaviour of exceptions based on BaseException is that they propagate over error handling. Currently, we'd need to do something hacky like inherit from SystemExit instead, basically relegating BaseException to something internal.

This issue was mentioned in the PR, and supposed to be adjusted for Python 3.9:

#13528 (review)

Unfortunately, it seems that was forgotten.

@CendioOssman CendioOssman added the type-bug An unexpected behavior, bug, or error label Aug 31, 2023
@github-project-automation github-project-automation bot moved this to Todo in asyncio Aug 31, 2023
@gvanrossum
Copy link
Member

I'm not entirely sure what behavior you expected. Can you describe the bug in terms of what you expected vs. what actually happened? And can you tell us in which Python version this changed? Is it new in 3.11? Finally, the issue you link to has several PRs associated with it. Which of those PRs do you think is the cause of the change in behavior?

@CendioOssman
Copy link
Contributor Author

Sorry about that. Let's see if I can make things more clear.

I'm not entirely sure what behavior you expected.

For the loop to stop. I.e. ShowStopper should propagate past loop.run_forever().

The de facto¹ standard has been that you derive from Exception for errors, and from BaseException if you want to force an unwind of the stack for other reasons. We've historically used this much like CancelledError in the referenced issue. In our case, it was to abort a deeply nested process on timeout by throwing a class TimeoutException(BaseException).

¹ I would appreciate it if Python could document this stronger. I.e. that try:/except: is an anti-pattern and you should be doing try:/except Exception:.

And can you tell us in which Python version this changed? Is it new in 3.11?

It started in Python 3.8.

To be clear, we have not yet encountered a real problem with this in asyncio. But it takes a long time for Python changes to propagate out to all distributions, so we want to be proactive when we see something that looks like it can cause issues for us down the line.

Finally, the issue you link to has several PRs associated with it. Which of those PRs do you think is the cause of the change in behavior?

My apologies. I followed things back from the commit and didn't notice that the issue had multiple PR. It's PR #13528.

I.e. changing this:

except Exception:

to

except (SystemExit, KeyboardInterrupt):
  raise
except BaseException:

This gives SystemExit and KeyboardInterrupt some special privilege over other things subclassed from BaseException.

The PR even mentions an approach that looks more appropriate to me:

except (Exception, asyncio.CancelledError):

@gvanrossum
Copy link
Member

The change is considered a feature. Previously, raising a base exception could violate internal invariants of the event loop machinery, because the event loop would only catch plain Exception. The changed policy handles all exceptions like, except KeyboardInterrupt and SystemExit -- those are now special, because it would break too much code to handle them the same way.

If you want to stop the event loop, use asyncio.get_running_loop().stop().

@gvanrossum gvanrossum closed this as not planned Won't fix, can't repro, duplicate, stale Sep 1, 2023
@github-project-automation github-project-automation bot moved this from Todo to Done in asyncio Sep 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-asyncio type-bug An unexpected behavior, bug, or error
Projects
Status: Done
Development

No branches or pull requests

3 participants