Skip to content

Update asyncio subprocess optional **kwargs. #9177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 18, 2022
Merged

Update asyncio subprocess optional **kwargs. #9177

merged 1 commit into from
Dec 18, 2022

Conversation

socketpair
Copy link
Contributor

Made strongly according to documentation (possible Sequence vs Iterable vs Collection bug).

But did not check.

@github-actions

This comment has been minimized.

1 similar comment
@github-actions

This comment has been minimized.

@socketpair
Copy link
Contributor Author

@JelleZijlstra

@srittau srittau added the topic: asyncio Asyncio-related issues label Nov 15, 2022
@JelleZijlstra JelleZijlstra self-requested a review November 15, 2022 16:57
Copy link
Member

@JelleZijlstra JelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These kwargs are delegated to subprocess.Popen, so it would be good to match the types exactly. It seems that 3.11 added another new kwarg, process_group: int | None = ..., so we should add a 3.11 branch to these stubs too.

pass_fds is Any in stdlib/subprocess.pyi, but from looking at the code Iterable[int] does appear to be the right type. Feel free to fix that type in this PR too.

@socketpair
Copy link
Contributor Author

@JelleZijlstra done

@github-actions

This comment has been minimized.

@socketpair
Copy link
Contributor Author

socketpair commented Nov 16, 2022

subprocess.py:

                    fds_to_keep = set(pass_fds)
                    fds_to_keep.add(errpipe_write)
                    self.pid = _posixsubprocess.fork_exec(
                            args, executable_list,
                            close_fds, tuple(sorted(map(int, fds_to_keep))),

Documentation: https://docs.python.org/3/library/subprocess.html

pass_fds is an optional sequence of file descriptors to keep open between the parent and child.

Yes, it's allowed to pass non-int. But I think it's a bug to pass non-int.

pypeeter code:

        options = dict()
        options['env'] = self.env
        if not self.dumpio:
            # discard stdout, it's never read in any case.
            options['stdout'] = subprocess.DEVNULL
            options['stderr'] = subprocess.STDOUT

        self.proc = subprocess.Popen(  # type: ignore
            self.cmd, **options, )

@socketpair
Copy link
Contributor Author

socketpair commented Nov 16, 2022

Since bool(some _generator) is always True, and code contains if pass_fds then it should be Collection.

@github-actions

This comment has been minimized.

@JelleZijlstra
Copy link
Member

The new error seems correct, though maybe mypy should listen to the type ignore: https://github.com/pyppeteer/pyppeteer/blob/7245ee0bccd97d43e45b801319dd6cbae860292a/pyppeteer/launcher.py#L148

extra_groups: None | Collection[str | int] = ...,
user: None | str | int = ...,
umask: int = ...,
process_group: str | int | None = ...,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

subprocess has this as just int | None, which looks correct based on the C code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont' understand what you wanted to say.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In subprocess.pyi, the type of the process_group parameter is int | None, not str | int | None. Based on my reading of the relevant C code, int | None appears to be correct.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@socketpair, you haven't addressed this comment here. Jelle is asking you to change the annotation for the process_group parameter (both for this function and all other functions that have this parameter) to be int | None instead of str | int | None. You need to either make the change or explain why you think that this would be incorrect.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really. it's my bug. Will fix in next commit.

@socketpair
Copy link
Contributor Author

What should I do to move this PR to merging ?

@socketpair
Copy link
Contributor Author

@JelleZijlstra done

@hauntsaninja
Copy link
Collaborator

I'm not seeing the change, did you forget to push?

@socketpair
Copy link
Contributor Author

@JelleZijlstra sorry for calling again. What should I change (what is wrong) in current PR? Please mark these changes somehow in order to distinguish from just speaking. Now, I will just rebase, no other changes.

@github-actions

This comment has been minimized.

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes until #9177 (comment) is addressed

@socketpair
Copy link
Contributor Author

I don't understand new Github review semantics. I just changed what you requested. I don't know if I need to tap something in GH

@AlexWaygood AlexWaygood dismissed their stale review December 12, 2022 18:19

Requested changes were made

@github-actions
Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

pyppeteer (https://github.com/pyppeteer/pyppeteer)
+ pyppeteer/launcher.py:149: error: Argument 2 to "Popen" has incompatible type "**Dict[str, Optional[Any]]"; expected "Collection[int]"  [arg-type]

@JelleZijlstra JelleZijlstra merged commit bff43b5 into python:main Dec 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: asyncio Asyncio-related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants