refine asyncio call_exception_handler context type

[graingert]

No description provided.

[JelleZijlstra]

I don't think we're ready to use PEP 655 in typeshed yet, unfortunately. #6301.

[sobolevn]

@JelleZijlstra looks like we can use intermediate total=False inheritance hack for this case.

[JelleZijlstra]

Let's see how much I can do in review suggestions

[JelleZijlstra]

Oops, I made pytype unhappy:

stdlib/asyncio/__init__.pyi (3.9): ParseError: 'total' allowed as classdef kwarg only for TypedDict subclasses

But I _Context is a TypedDict subclass. I filed google/pytype#1122.

[srittau]

+1 for more TypedDicts. A few things I noticed below.

[srittau]

Looking forward to PEP 655.

default_exception_handler() also adds a handle_trackback field to the context:

https://github.com/python/cpython/blob/0a222db2bca63070f429c0e613707da1bdfaf0e0/Lib/asyncio/base_events.py#L1732-L1733

[srittau]

And Handle._run() adds source_traceback under some conditions:

https://github.com/python/cpython/blob/0a222db2bca63070f429c0e613707da1bdfaf0e0/Lib/asyncio/events.py#L92-L93

[srittau]

Is this really required? At least default_exception_handler() uses .get() to access it:

https://github.com/python/cpython/blob/0a222db2bca63070f429c0e613707da1bdfaf0e0/Lib/asyncio/base_events.py#L1719

[hauntsaninja]

This is quite disruptive and I don't think we should merge. Because this type doesn't exist at runtime, I think it means in practice that any time you're not passing a dict literal, you'll get a type error, with no good way to refer to the type.

[AlexWaygood]

This no longer breaks pytype, so I've removed the "deferred" label. We now just need to decide whether it's a good idea or not.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

anyio (https://github.com/agronholm/anyio)
+ src/anyio/_backends/_asyncio.py:1910: error: Argument 1 to "set_exception_handler" of "AbstractEventLoop" has incompatible type "Callable[[AbstractEventLoop, Dict[str, Any]], None]"; expected "Optional[Callable[[AbstractEventLoop, _Context], Any]]"  [arg-type]

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/connector.py:108: error: Argument 1 to "call_exception_handler" of "AbstractEventLoop" has incompatible type "Dict[str, object]"; expected "_Context"  [arg-type]
+ aiohttp/connector.py:258: error: Argument 1 to "call_exception_handler" of "AbstractEventLoop" has incompatible type "Dict[str, object]"; expected "_Context"  [arg-type]
+ aiohttp/client.py:306: error: Argument 1 to "call_exception_handler" of "AbstractEventLoop" has incompatible type "Dict[str, object]"; expected "_Context"  [arg-type]
+ aiohttp/web.py:455: error: Incompatible types (expression has type "Optional[BaseException]", TypedDict item "exception" has type "BaseException")  [typeddict-item]

core (https://github.com/home-assistant/core)
+ homeassistant/runner.py:67: error: Argument 1 to "set_exception_handler" of "AbstractEventLoop" has incompatible type "Callable[[Any, Dict[str, Any]], None]"; expected "Optional[Callable[[AbstractEventLoop, _Context], Any]]"  [arg-type]
+ homeassistant/runner.py:156: error: Incompatible types (expression has type "Optional[BaseException]", TypedDict item "exception" has type "BaseException")  [typeddict-item]

boostedblob (https://github.com/hauntsaninja/boostedblob)
+ boostedblob/globals.py:231: error: Argument 1 to "default_exception_handler" of "AbstractEventLoop" has incompatible type "Dict[str, Any]"; expected "_Context"
+ boostedblob/globals.py:233: error: Argument 1 to "set_exception_handler" of "AbstractEventLoop" has incompatible type "Callable[[AbstractEventLoop, Dict[str, Any]], None]"; expected "Optional[Callable[[AbstractEventLoop, _Context], Any]]"

aioredis (https://github.com/aio-libs/aioredis)
+ aioredis/client.py:1076: error: Argument 1 to "call_exception_handler" of "AbstractEventLoop" has incompatible type "Dict[str, object]"; expected "_Context"  [arg-type]

python-chess (https://github.com/niklasf/python-chess)
+ chess/engine.py:1261: error: Incompatible types (expression has type "Optional[SubprocessTransport]", TypedDict item "transport" has type "BaseTransport")

dragonchain (https://github.com/dragonchain/dragonchain)
+ dragonchain/broadcast_processor/broadcast_processor.py:367:42: error: Argument 1 to "set_exception_handler" of "AbstractEventLoop" has incompatible type "Callable[[AbstractEventLoop, Dict[Any, Any]], None]"; expected "Optional[Callable[[AbstractEventLoop, _Context], Any]]"
+ dragonchain/contract_invoker/contract_invoker.py:137:42: error: Argument 1 to "set_exception_handler" of "AbstractEventLoop" has incompatible type "Callable[[AbstractEventLoop, Dict[Any, Any]], None]"; expected "Optional[Callable[[AbstractEventLoop, _Context], Any]]"

[AlexWaygood]

I'm closing this PR: as @hauntsaninja says, this would be quite disruptive, and it would be complicated trying to access this TypedDict at runtime. I don't think we have enough consensus to make such a disruptive change; @srittau's review from February also has not been addressed.

The asyncio maintainers don't seem averse to typing, however: some of the recent additions have been quite heavily typed. Perhaps a better way forward would be to file an issue over at CPython asking for this TypedDict to be added to the runtime. Then users would have an easy way of accessing the TypedDict at runtime, and so the concerns about disruption wouldn't be nearly so great.