Infer correct types with overloads of Type[Guard | Is]

[sobolevn]

Closes #17579

Consider this as a prototype, because I understand that there might be a lot of extra work to get this right.

However, this does solve this problem in the original issue.

[sobolevn]

Happy New Year everyone! I want to raise attention to this PR :)

[JukkaL]

Sorry for the slow response! Looks good to me, just a few minor comments.

[sobolevn]

Done!

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

discord.py (https://github.com/Rapptz/discord.py)
- discord/ui/select.py:1072: error: "Callable[[V, Interaction[Any], BaseSelectT], Coroutine[Any, Any, Any]]" has no attribute "__discord_ui_model_type__"  [attr-defined]
+ discord/ui/select.py:1072: error: Item "function" of "Callable[[V, Interaction[Any], BaseSelectT], CoroutineType[Any, Any, Any]] | Callable[..., CoroutineType[Any, Any, Any]]" has no attribute "__discord_ui_model_type__"  [union-attr]
- discord/ui/select.py:1073: error: "Callable[[V, Interaction[Any], BaseSelectT], Coroutine[Any, Any, Any]]" has no attribute "__discord_ui_model_kwargs__"  [attr-defined]
+ discord/ui/select.py:1073: error: Item "function" of "Callable[[V, Interaction[Any], BaseSelectT], CoroutineType[Any, Any, Any]] | Callable[..., CoroutineType[Any, Any, Any]]" has no attribute "__discord_ui_model_kwargs__"  [union-attr]
- discord/ui/select.py:1082: error: "Callable[[V, Interaction[Any], BaseSelectT], Coroutine[Any, Any, Any]]" has no attribute "__discord_ui_model_kwargs__"  [attr-defined]
+ discord/ui/select.py:1082: error: Item "function" of "Callable[[V, Interaction[Any], BaseSelectT], CoroutineType[Any, Any, Any]] | Callable[..., CoroutineType[Any, Any, Any]]" has no attribute "__discord_ui_model_kwargs__"  [union-attr]
- discord/ui/select.py:1084: error: "Callable[[V, Interaction[Any], BaseSelectT], Coroutine[Any, Any, Any]]" has no attribute "__discord_ui_model_kwargs__"  [attr-defined]
+ discord/ui/select.py:1084: error: Item "function" of "Callable[[V, Interaction[Any], BaseSelectT], CoroutineType[Any, Any, Any]] | Callable[..., CoroutineType[Any, Any, Any]]" has no attribute "__discord_ui_model_kwargs__"  [union-attr]
- discord/ui/select.py:1100: error: "Callable[[V, Interaction[Any], BaseSelectT], Coroutine[Any, Any, Any]]" has no attribute "__discord_ui_model_kwargs__"  [attr-defined]
+ discord/ui/select.py:1100: error: Item "function" of "Callable[[V, Interaction[Any], BaseSelectT], CoroutineType[Any, Any, Any]] | Callable[..., CoroutineType[Any, Any, Any]]" has no attribute "__discord_ui_model_kwargs__"  [union-attr]
+ discord/ext/tasks/__init__.py:631: error: Incompatible return value type (got "Callable[[Any, BaseException], CoroutineType[Any, Any, Any]] | Callable[..., CoroutineType[Any, Any, Any]]", expected "ET")  [return-value]

steam.py (https://github.com/Gobot1234/steam.py)
- steam/ext/commands/commands.py:331: error: Incompatible types in assignment (expression has type "InvokeT", variable has type "None")  [assignment]
- steam/ext/commands/commands.py:339: error: Incompatible types in assignment (expression has type "InvokeT", variable has type "None")  [assignment]
+ steam/ext/commands/commands.py:324: error: Incompatible return value type (got "Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, None]] | Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, Any]]", expected "ErrT")  [return-value]
+ steam/ext/commands/commands.py:331: error: Incompatible types in assignment (expression has type "Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, None]] | Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, Any]]", variable has type "None")  [assignment]
+ steam/ext/commands/commands.py:332: error: Incompatible return value type (got "Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, None]] | Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, Any]]", expected "InvokeT")  [return-value]
+ steam/ext/commands/commands.py:339: error: Incompatible types in assignment (expression has type "Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, None]] | Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, Any]]", variable has type "None")  [assignment]
+ steam/ext/commands/commands.py:340: error: Incompatible return value type (got "Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, None]] | Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, Any]]", expected "InvokeT")  [return-value]
- steam/ext/commands/help.py:138: note:          ErrT
+ steam/ext/commands/help.py:138: note:          Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, None]] | Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, Any]]
+ steam/ext/commands/cog.py:180: error: Incompatible return value type (got "Callable[[Any | None, VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, None]] | Callable[[Any | None, VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, Any]] | Callable[..., CoroutineType[Any, Any, Any]]", expected "F")  [return-value]
- steam/ext/commands/bot.py:467: error: Incompatible types in assignment (expression has type "BotInvokeT", variable has type "None")  [assignment]
- steam/ext/commands/bot.py:475: error: Incompatible types in assignment (expression has type "BotInvokeT", variable has type "None")  [assignment]
+ steam/ext/commands/bot.py:467: error: Incompatible types in assignment (expression has type "Callable[[Context[Any]], CoroutineType[Any, Any, None]] | Callable[[Context[Any]], CoroutineType[Any, Any, Any]] | Callable[..., CoroutineType[Any, Any, Any]]", variable has type "None")  [assignment]
+ steam/ext/commands/bot.py:468: error: Incompatible return value type (got "Callable[[Context[Any]], CoroutineType[Any, Any, None]] | Callable[[Context[Any]], CoroutineType[Any, Any, Any]] | Callable[..., CoroutineType[Any, Any, Any]]", expected "BotInvokeT")  [return-value]
+ steam/ext/commands/bot.py:475: error: Incompatible types in assignment (expression has type "Callable[[Context[Any]], CoroutineType[Any, Any, None]] | Callable[[Context[Any]], CoroutineType[Any, Any, Any]] | Callable[..., CoroutineType[Any, Any, Any]]", variable has type "None")  [assignment]
+ steam/ext/commands/bot.py:476: error: Incompatible return value type (got "Callable[[Context[Any]], CoroutineType[Any, Any, None]] | Callable[[Context[Any]], CoroutineType[Any, Any, Any]] | Callable[..., CoroutineType[Any, Any, Any]]", expected "BotInvokeT")  [return-value]

[JukkaL]

Ah one more thing before merging: Is the mypy primer output as expected?

[sobolevn]

discord.py

Yes, just wording changes.

steam.py

It has some new violations. Exploring:

- steam/ext/commands/commands.py:331: error: Incompatible types in assignment (expression has type "InvokeT", variable has type "None")  [assignment]
- steam/ext/commands/commands.py:339: error: Incompatible types in assignment (expression has type "InvokeT", variable has type "None")  [assignment]
+ steam/ext/commands/commands.py:324: error: Incompatible return value type (got "Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, None]] | Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, Any]]", expected "ErrT")  [return-value]
+ steam/ext/commands/commands.py:331: error: Incompatible types in assignment (expression has type "Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, None]] | Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, Any]]", variable has type "None")  [assignment]
+ steam/ext/commands/commands.py:332: error: Incompatible return value type (got "Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, None]] | Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, Any]]", expected "InvokeT")  [return-value]
+ steam/ext/commands/commands.py:339: error: Incompatible types in assignment (expression has type "Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, None]] | Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, Any]]", variable has type "None")  [assignment]
+ steam/ext/commands/commands.py:340: error: Incompatible return value type (got "Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, None]] | Callable[[VarArg(Any), KwArg(Any)], CoroutineType[Any, Any, Any]]", expected "InvokeT")  [return-value]

https://github.com/Gobot1234/steam.py/blob/d1f14354cec5dda30f3cb1fa28906eaf6dc81db4/steam/ext/commands/commands.py#L324

I tried to create a local repro:

from typing import TypeVar, Callable, Coroutine, Generic, ParamSpec, TypeAlias, Any
import inspect


P = ParamSpec("P", default=...)
R = TypeVar("R", default=Any, covariant=True)


_T = TypeVar("_T")
Coro: TypeAlias = Coroutine[Any, Any, _T]
MaybeCoroFunc: TypeAlias = Callable[P, R | Coro[R]]
CoroFunc: TypeAlias = Callable[P, Coro[R]]
ErrT = TypeVar("ErrT", bound=CoroFunc[[int, Exception], None] | CoroFunc[[int, int, Exception], None])

class Command(Generic[_T]):
    def error(self, coro: ErrT) -> ErrT:
        if not inspect.iscoroutinefunction(coro):
            raise TypeError(f"Error handler for {self}")
        self.on_error = coro
        return coro

And it works correctly. But, mypy-primer still fails for unknown to me reason.

steam.py has Found 973 errors in 86 files (checked 134 source files) with just the latest mypy. So, I cannot be sure what is going on there.

So, I think that it is safe to ignore.
Maybe we should even delete this project from primer. I raised a discussion: Gobot1234/steam.py#597

I think it is safe to be merged. I will wait a bit for some further feedback and merge.

Thanks a lot, @JukkaL! 🤝