Replace various Incompletes in stdlib #11673

srittau · 2024-03-30T13:06:16Z

No description provided.

srittau · 2024-03-30T13:08:57Z

stdlib/importlib/resources/simple.pyi

+        def open(
+            self,
+            mode: OpenTextMode = "r",
+            *,
+            encoding: str | None = ...,
+            errors: str | None = ...,
+            newline: str | None = ...,
+            line_buffering: bool = ...,
+            write_through: bool = ...,
+        ) -> TextIOWrapper: ...
        @overload
-        def open(self, mode: OpenBinaryMode, *args: Unused, **kwargs: Unused) -> BinaryIO: ...
+        def open(
+            self,
+            mode: OpenTextMode,
+            buffer: _WrappedBuffer,
+            encoding: str | None = ...,
+            errors: str | None = ...,
+            newline: str | None = ...,
+            line_buffering: bool = ...,
+            write_through: bool = ...,
+        ) -> TextIOWrapper: ...


For text modes, the arguments are just passed on to TextIOWrapper.__init__(). That said, I'm not sure if passing a buffer here is really a good idea. But then, the implementation of this method is strange anyway, creating a stream when using the text mode, but then not passing that stream to TextIOWrapper ...

It was indeed a CPython bug: python/cpython#111775.

srittau · 2024-03-30T13:09:47Z

stdlib/importlib/resources/simple.pyi

        @overload
-        def open(self, mode: str, *args: Incomplete, **kwargs: Incomplete) -> IO[Any]: ...
+        def open(self, mode: OpenBinaryMode) -> BinaryIO: ...


While args and kwargs are just ignored in this case, passing any further arguments is potentially a bug, which is why I removed the arguments and their Unused annotations.

srittau · 2024-03-30T13:10:49Z

stdlib/typing.pyi

+            def __typing_subst__(self, arg: Any) -> Any: ...
+            def __typing_prepare_subst__(self, alias: Any, args: Any) -> tuple[Any, ...]: ...


These dunders are unfortunately completely undocumented, but it seems that only type forms etc. are passed through, so Any is probably the most appropriate annotation for the time being.

Yeah, these are very much implementation details that users should steer clear of... Possibly we could/should just remove them from the stubs and allowlist them. But I'm also fine with just using Any.

stdlib/importlib/resources/simple.pyi

Co-authored-by: Alex Waygood <[email protected]>

AlexWaygood · 2024-04-01T13:34:01Z

stdlib/importlib/resources/simple.pyi

+        def open(
+            self,
+            mode: Literal["r"] = "r",
+            encoding: str | None = None,
+            errors: str | None = None,
+            newline: str | None = None,
+            line_buffering: bool = False,
+            write_through: bool = False,
+        ) -> TextIOWrapper: ...
        @overload
-        def open(self, mode: OpenBinaryMode, *args: Unused, **kwargs: Unused) -> BinaryIO: ...
+        def open(self, mode: Literal["rb"]) -> BinaryIO: ...


These overloads are a lot more selective than they used to be. On main, the first overload matches if any of these strings is passed in as a literal string:

typeshed/stdlib/_typeshed/__init__.pyi

Lines 164 to 199 in 0cea0bc

OpenTextModeUpdating: TypeAlias = Literal[

"r+",

"+r",

"rt+",

"r+t",

"+rt",

"tr+",

"t+r",

"+tr",

"w+",

"+w",

"wt+",

"w+t",

"+wt",

"tw+",

"t+w",

"+tw",

"a+",

"+a",

"at+",

"a+t",

"+at",

"ta+",

"t+a",

"+ta",

"x+",

"+x",

"xt+",

"x+t",

"+xt",

"tx+",

"t+x",

"+tx",

]

OpenTextModeWriting: TypeAlias = Literal["w", "wt", "tw", "a", "at", "ta", "x", "xt", "tx"]

OpenTextModeReading: TypeAlias = Literal["r", "rt", "tr", "U", "rU", "Ur", "rtU", "rUt", "Urt", "trU", "tUr", "Utr"]

And similarly, the second overload on main matches if any of these strings is passed in as a literal string:

typeshed/stdlib/_typeshed/__init__.pyi

Lines 201 to 228 in 0cea0bc

OpenBinaryModeUpdating: TypeAlias = Literal[

"rb+",

"r+b",

"+rb",

"br+",

"b+r",

"+br",

"wb+",

"w+b",

"+wb",

"bw+",

"b+w",

"+bw",

"ab+",

"a+b",

"+ab",

"ba+",

"b+a",

"+ba",

"xb+",

"x+b",

"+xb",

"bx+",

"b+x",

"+bx",

]

OpenBinaryModeWriting: TypeAlias = Literal["wb", "bw", "ab", "ba", "xb", "bx"]

OpenBinaryModeReading: TypeAlias = Literal["rb", "br", "rbU", "rUb", "Urb", "brU", "bUr", "Ubr"]

What's the reason for changing it so that the first overload only matches if a literal "r" is passed in, and so that the second overload only matches if a literal "rb" is passed in?

It's now matching the documentation of Traversable. All other modes make no sense and are likely a bug: https://github.com/python/cpython/blob/dd44ab994b7262f0704d64996e0a1bc37b233407/Lib/importlib/resources/simple.py#L88-L92

(You could argue for allowing br, but there's really no case for using this non-standard ordering.)

In theory, we could also allow "" and "b", but this is also non-standard according to the docs.

stdlib/multiprocessing/util.pyi

AlexWaygood

Looks good other than my two questions above

Co-authored-by: Alex Waygood <[email protected]>

AlexWaygood

Thanks!

github-actions · 2024-04-01T14:19:02Z

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

Replace various Incompletes in stdlib

14cfa62

srittau commented Mar 30, 2024

View reviewed changes

srittau and others added 4 commits March 30, 2024 14:11

Add missing Generic marker

c921c55

Add an explanatory comment

b552350

ResourceHandle.open() had a bug upstream

d51b3de

[pre-commit.ci] auto fixes from pre-commit.com hooks

eefbc2d

This comment has been minimized.

Sign in to view

srittau added 2 commits March 30, 2024 14:26

Add defaults to TextIOWrapper

2c76845

Match ResourceHandle.open() to ABC

3e0bb77

This comment has been minimized.

Sign in to view

AlexWaygood reviewed Apr 1, 2024

View reviewed changes

stdlib/importlib/resources/simple.pyi Show resolved Hide resolved

srittau and others added 3 commits April 1, 2024 14:29

Update stdlib/importlib/resources/simple.pyi

48d2464

Co-authored-by: Alex Waygood <[email protected]>

Merge branch 'main' into typing-incompletes

3d98255

[pre-commit.ci] auto fixes from pre-commit.com hooks

35439f3

This comment has been minimized.

Sign in to view

Update simple.pyi

d60e8c9

This comment has been minimized.

Sign in to view

Merge branch 'main' into typing-incompletes

def5de0

This comment has been minimized.

Sign in to view

AlexWaygood reviewed Apr 1, 2024

View reviewed changes

stdlib/multiprocessing/util.pyi Outdated Show resolved Hide resolved

AlexWaygood reviewed Apr 1, 2024

View reviewed changes

srittau and others added 2 commits April 1, 2024 16:08

Make _R covariant

a0580f3

Co-authored-by: Alex Waygood <[email protected]>

Fix typevar name

b77a5cb

AlexWaygood approved these changes Apr 1, 2024

View reviewed changes

AlexWaygood merged commit 630b49a into python:main Apr 1, 2024

srittau deleted the typing-incompletes branch April 1, 2024 14:21

		def __typing_subst__(self, arg: Any) -> Any: ...
		def __typing_prepare_subst__(self, alias: Any, args: Any) -> tuple[Any, ...]: ...

	OpenTextModeUpdating: TypeAlias = Literal[
	"r+",
	"+r",
	"rt+",
	"r+t",
	"+rt",
	"tr+",
	"t+r",
	"+tr",
	"w+",
	"+w",
	"wt+",
	"w+t",
	"+wt",
	"tw+",
	"t+w",
	"+tw",
	"a+",
	"+a",
	"at+",
	"a+t",
	"+at",
	"ta+",
	"t+a",
	"+ta",
	"x+",
	"+x",
	"xt+",
	"x+t",
	"+xt",
	"tx+",
	"t+x",
	"+tx",
	]
	OpenTextModeWriting: TypeAlias = Literal["w", "wt", "tw", "a", "at", "ta", "x", "xt", "tx"]
	OpenTextModeReading: TypeAlias = Literal["r", "rt", "tr", "U", "rU", "Ur", "rtU", "rUt", "Urt", "trU", "tUr", "Utr"]

	OpenBinaryModeUpdating: TypeAlias = Literal[
	"rb+",
	"r+b",
	"+rb",
	"br+",
	"b+r",
	"+br",
	"wb+",
	"w+b",
	"+wb",
	"bw+",
	"b+w",
	"+bw",
	"ab+",
	"a+b",
	"+ab",
	"ba+",
	"b+a",
	"+ba",
	"xb+",
	"x+b",
	"+xb",
	"bx+",
	"b+x",
	"+bx",
	]
	OpenBinaryModeWriting: TypeAlias = Literal["wb", "bw", "ab", "ba", "xb", "bx"]
	OpenBinaryModeReading: TypeAlias = Literal["rb", "br", "rbU", "rUb", "Urb", "brU", "bUr", "Ubr"]

Uh oh!

Replace various Incompletes in stdlib #11673

Replace various Incompletes in stdlib #11673

Uh oh!

Conversation

srittau commented Mar 30, 2024

Uh oh!

srittau Mar 30, 2024

Choose a reason for hiding this comment

Uh oh!

srittau Mar 30, 2024

Choose a reason for hiding this comment

Uh oh!

srittau Mar 30, 2024

Choose a reason for hiding this comment

Uh oh!

srittau Mar 30, 2024

Choose a reason for hiding this comment

Uh oh!

AlexWaygood Mar 30, 2024

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

AlexWaygood Apr 1, 2024

Choose a reason for hiding this comment

Uh oh!

srittau Apr 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

srittau Apr 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlexWaygood Apr 1, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AlexWaygood left a comment

Choose a reason for hiding this comment

Uh oh!

AlexWaygood left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 1, 2024

Uh oh!

Uh oh!

srittau Apr 1, 2024 •

edited

Loading

srittau Apr 1, 2024 •

edited

Loading