Skip to content

Replace various Incompletes in stdlib #11673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Apr 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions stdlib/importlib/resources/simple.pyi
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import abc
import sys
from _typeshed import Incomplete, OpenBinaryMode, OpenTextMode, Unused
from collections.abc import Iterator
from io import TextIOWrapper
from typing import IO, Any, BinaryIO, Literal, NoReturn, overload
Expand Down Expand Up @@ -28,11 +27,19 @@ if sys.version_info >= (3, 11):
def is_file(self) -> Literal[True]: ...
def is_dir(self) -> Literal[False]: ...
@overload
def open(self, mode: OpenTextMode = "r", *args, **kwargs) -> TextIOWrapper: ...
def open(
self,
mode: Literal["r"] = "r",
encoding: str | None = None,
errors: str | None = None,
newline: str | None = None,
line_buffering: bool = False,
write_through: bool = False,
) -> TextIOWrapper: ...
@overload
def open(self, mode: OpenBinaryMode, *args: Unused, **kwargs: Unused) -> BinaryIO: ...
def open(self, mode: Literal["rb"]) -> BinaryIO: ...
Comment on lines +30 to +40
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These overloads are a lot more selective than they used to be. On main, the first overload matches if any of these strings is passed in as a literal string:

OpenTextModeUpdating: TypeAlias = Literal[
"r+",
"+r",
"rt+",
"r+t",
"+rt",
"tr+",
"t+r",
"+tr",
"w+",
"+w",
"wt+",
"w+t",
"+wt",
"tw+",
"t+w",
"+tw",
"a+",
"+a",
"at+",
"a+t",
"+at",
"ta+",
"t+a",
"+ta",
"x+",
"+x",
"xt+",
"x+t",
"+xt",
"tx+",
"t+x",
"+tx",
]
OpenTextModeWriting: TypeAlias = Literal["w", "wt", "tw", "a", "at", "ta", "x", "xt", "tx"]
OpenTextModeReading: TypeAlias = Literal["r", "rt", "tr", "U", "rU", "Ur", "rtU", "rUt", "Urt", "trU", "tUr", "Utr"]

And similarly, the second overload on main matches if any of these strings is passed in as a literal string:

OpenBinaryModeUpdating: TypeAlias = Literal[
"rb+",
"r+b",
"+rb",
"br+",
"b+r",
"+br",
"wb+",
"w+b",
"+wb",
"bw+",
"b+w",
"+bw",
"ab+",
"a+b",
"+ab",
"ba+",
"b+a",
"+ba",
"xb+",
"x+b",
"+xb",
"bx+",
"b+x",
"+bx",
]
OpenBinaryModeWriting: TypeAlias = Literal["wb", "bw", "ab", "ba", "xb", "bx"]
OpenBinaryModeReading: TypeAlias = Literal["rb", "br", "rbU", "rUb", "Urb", "brU", "bUr", "Ubr"]

What's the reason for changing it so that the first overload only matches if a literal "r" is passed in, and so that the second overload only matches if a literal "rb" is passed in?

Copy link
Collaborator Author

@srittau srittau Apr 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's now matching the documentation of Traversable. All other modes make no sense and are likely a bug: https://github.com/python/cpython/blob/dd44ab994b7262f0704d64996e0a1bc37b233407/Lib/importlib/resources/simple.py#L88-L92

(You could argue for allowing br, but there's really no case for using this non-standard ordering.)

Copy link
Collaborator Author

@srittau srittau Apr 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we could also allow "" and "b", but this is also non-standard according to the docs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okiedokie

@overload
def open(self, mode: str, *args: Incomplete, **kwargs) -> IO[Any]: ...
def open(self, mode: str) -> IO[Any]: ...
def joinpath(self, name: Never) -> NoReturn: ... # type: ignore[override]

class ResourceContainer(Traversable, metaclass=abc.ABCMeta):
Expand Down
10 changes: 5 additions & 5 deletions stdlib/io.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -179,11 +179,11 @@ class TextIOWrapper(TextIOBase, TextIO): # type: ignore[misc] # incompatible d
def __init__(
self,
buffer: _WrappedBuffer,
encoding: str | None = ...,
errors: str | None = ...,
newline: str | None = ...,
line_buffering: bool = ...,
write_through: bool = ...,
encoding: str | None = None,
errors: str | None = None,
newline: str | None = None,
line_buffering: bool = False,
write_through: bool = False,
) -> None: ...
# Equals the "buffer" argument passed in to the constructor.
@property
Expand Down
4 changes: 2 additions & 2 deletions stdlib/multiprocessing/resource_tracker.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ __all__ = ["ensure_running", "register", "unregister"]
class ResourceTracker:
def getfd(self) -> int | None: ...
def ensure_running(self) -> None: ...
def register(self, name: Sized, rtype) -> None: ...
def unregister(self, name: Sized, rtype) -> None: ...
def register(self, name: Sized, rtype: str) -> None: ...
def unregister(self, name: Sized, rtype: str) -> None: ...

_resource_tracker: ResourceTracker
ensure_running = _resource_tracker.ensure_running
Expand Down
31 changes: 25 additions & 6 deletions stdlib/multiprocessing/util.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import threading
from _typeshed import ConvertibleToInt, Incomplete, Unused
from collections.abc import Callable, Iterable, Mapping, MutableMapping, Sequence
from logging import Logger, _Level as _LoggingLevel
from typing import Any
from typing import Any, Generic, TypeVar, overload

__all__ = [
"sub_debug",
Expand All @@ -22,6 +22,9 @@ __all__ = [
"SUBWARNING",
]

_T = TypeVar("_T")
_R_co = TypeVar("_R_co", default=Any, covariant=True)

NOTSET: int
SUBDEBUG: int
DEBUG: int
Expand All @@ -42,13 +45,29 @@ def is_abstract_socket_namespace(address: str | bytes | None) -> bool: ...
abstract_sockets_supported: bool

def get_temp_dir() -> str: ...
def register_after_fork(obj, func: Callable[[Incomplete], object]) -> None: ...
def register_after_fork(obj: _T, func: Callable[[_T], object]) -> None: ...

class Finalize:
class Finalize(Generic[_R_co]):
# "args" and "kwargs" are passed as arguments to "callback".
@overload
def __init__(
self,
obj: None,
callback: Callable[..., _R_co],
*,
args: Sequence[Any] = (),
kwargs: Mapping[str, Any] | None = None,
exitpriority: int,
) -> None: ...
@overload
def __init__(
self, obj: None, callback: Callable[..., _R_co], args: Sequence[Any], kwargs: Mapping[str, Any] | None, exitpriority: int
) -> None: ...
@overload
def __init__(
self,
obj: Incomplete | None,
callback: Callable[..., Incomplete],
obj: Any,
callback: Callable[..., _R_co],
args: Sequence[Any] = (),
kwargs: Mapping[str, Any] | None = None,
exitpriority: int | None = None,
Expand All @@ -59,7 +78,7 @@ class Finalize:
_finalizer_registry: MutableMapping[Incomplete, Incomplete] = {},
sub_debug: Callable[..., object] = ...,
getpid: Callable[[], int] = ...,
): ...
) -> _R_co: ...
def cancel(self) -> None: ...
def still_active(self) -> bool: ...

Expand Down
8 changes: 4 additions & 4 deletions stdlib/typing.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ class TypeVar:
def __or__(self, right: Any) -> _SpecialForm: ...
def __ror__(self, left: Any) -> _SpecialForm: ...
if sys.version_info >= (3, 11):
def __typing_subst__(self, arg): ...
def __typing_subst__(self, arg: Any) -> Any: ...

# Used for an undocumented mypy feature. Does not exist at runtime.
_promote = object()
Expand Down Expand Up @@ -221,7 +221,7 @@ if sys.version_info >= (3, 11):
def __init__(self, name: str) -> None: ...
def __iter__(self) -> Any: ...
def __typing_subst__(self, arg: Never) -> Never: ...
def __typing_prepare_subst__(self, alias, args): ...
def __typing_prepare_subst__(self, alias: Any, args: Any) -> tuple[Any, ...]: ...

if sys.version_info >= (3, 10):
@final
Expand Down Expand Up @@ -270,8 +270,8 @@ if sys.version_info >= (3, 10):
@property
def kwargs(self) -> ParamSpecKwargs: ...
if sys.version_info >= (3, 11):
def __typing_subst__(self, arg): ...
def __typing_prepare_subst__(self, alias, args): ...
def __typing_subst__(self, arg: Any) -> Any: ...
def __typing_prepare_subst__(self, alias: Any, args: Any) -> tuple[Any, ...]: ...
Comment on lines +273 to +274
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These dunders are unfortunately completely undocumented, but it seems that only type forms etc. are passed through, so Any is probably the most appropriate annotation for the time being.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, these are very much implementation details that users should steer clear of... Possibly we could/should just remove them from the stubs and allowlist them. But I'm also fine with just using Any.


def __or__(self, right: Any) -> _SpecialForm: ...
def __ror__(self, left: Any) -> _SpecialForm: ...
Expand Down
2 changes: 1 addition & 1 deletion stdlib/typing_extensions.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -413,7 +413,7 @@ class TypeVar:
def __or__(self, right: Any) -> _SpecialForm: ...
def __ror__(self, left: Any) -> _SpecialForm: ...
if sys.version_info >= (3, 11):
def __typing_subst__(self, arg): ...
def __typing_subst__(self, arg: Any) -> Any: ...

@final
class ParamSpec:
Expand Down
2 changes: 1 addition & 1 deletion stdlib/xml/dom/xmlbuilder.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ class DOMBuilder:
def supportsFeature(self, name: str) -> bool: ...
def canSetFeature(self, name: str, state: int) -> bool: ...
# getFeature could return any attribute from an instance of `Options`
def getFeature(self, name: str): ...
def getFeature(self, name: str) -> Any: ...
def parseURI(self, uri: str) -> ExpatBuilder | ExpatBuilderNS: ...
def parse(self, input: DOMInputSource) -> ExpatBuilder | ExpatBuilderNS: ...
# `input` and `cnode` argtypes for `parseWithContext` are unknowable
Expand Down