Skip to content

tarfile.open(): Protocols for pipe modes #12182

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
srittau opened this issue Jun 21, 2024 · 0 comments · Fixed by #13130
Closed

tarfile.open(): Protocols for pipe modes #12182

srittau opened this issue Jun 21, 2024 · 0 comments · Fixed by #13130

Comments

@srittau
Copy link
Collaborator

srittau commented Jun 21, 2024 •
edited
Loading

The pipe modes for tarfile.open() require only a simplified file object, not the full protocol. From the documentation:

For special purposes, there is a second format for mode: 'filemode|[compression]'. tarfile.open() will return a TarFile object that processes its data as a stream of blocks. No random seeking will be done on the file. If given, fileobj may be any object that has a read() or write() method (depending on the mode) that works with bytes. [...]

But currently, adding support for those modes is not possible, since mypy can't process string literals that contain a pipe character: python/mypy#16367. This will be fixed in mypy 1.11.
@AlexWaygood AlexWaygood mentioned this issue Jun 21, 2024
Merged
tsibley added a commit to nextstrain/cli that referenced this issue Sep 5, 2024
@tsibley
runner.conda: Ignore newly failing type check for tarfile.open()
c79a8f3 
This started failing recently with newer typeshed stubs for
tarfile.open() (and/or newer stubs for requests.Response?).

mypy fails with:

    nextstrain/cli/runner/conda.py:220: error: Argument "fileobj" to "open" has incompatible type "Union[HTTPResponse, Any]"; expected "Optional[IO[bytes]]"

and pyright with:

    nextstrain/cli/runner/conda.py:220:14 - error: No overloads for "open" match the provided arguments (reportCallIssue)
    nextstrain/cli/runner/conda.py:220:58 - error: Argument of type "Literal['r|*']" cannot be assigned to parameter "mode" of type "_FileCreationModes" in function "open"
      Type "Literal['r|*']" is not assignable to type "_FileCreationModes"
        "Literal['r|*']" is not assignable to type "Literal['a']"
        "Literal['r|*']" is not assignable to type "Literal['w']"
        "Literal['r|*']" is not assignable to type "Literal['x']" (reportArgumentType)

I'm guessing the difference is because they match the different
overloads of tarfile.open().

I dug into it a little¹, but the stubs here seem a bit scattershot at
the moment so I stopped and went with an ignore.  An alternative fix is
casting response.raw to BinaryIO, but that seemed uglier and worse than
an ignore.

¹ See <python/typeshed#12117> (implicated)
  and <python/typeshed#12181> (open, may fix?)
  and <python/typeshed#12182> (open, may fix?)
tsibley added a commit to nextstrain/cli that referenced this issue Sep 5, 2024
@tsibley
runner.conda: Ignore newly failing type check for tarfile.open()
73ed6cb 
This started failing recently with newer typeshed stubs for
tarfile.open() (and/or newer stubs for requests.Response?).

mypy fails with:

    nextstrain/cli/runner/conda.py:220: error: Argument "fileobj" to "open" has incompatible type "Union[HTTPResponse, Any]"; expected "Optional[IO[bytes]]"

and pyright with:

    nextstrain/cli/runner/conda.py:220:14 - error: No overloads for "open" match the provided arguments (reportCallIssue)
    nextstrain/cli/runner/conda.py:220:58 - error: Argument of type "Literal['r|*']" cannot be assigned to parameter "mode" of type "_FileCreationModes" in function "open"
      Type "Literal['r|*']" is not assignable to type "_FileCreationModes"
        "Literal['r|*']" is not assignable to type "Literal['a']"
        "Literal['r|*']" is not assignable to type "Literal['w']"
        "Literal['r|*']" is not assignable to type "Literal['x']" (reportArgumentType)

I'm guessing the difference is because they match the different
overloads of tarfile.open().

I dug into it a little¹, but the stubs here seem a bit scattershot at
the moment so I stopped and went with an ignore.  An alternative fix is
casting response.raw to BinaryIO, but that seemed uglier and worse than
an ignore.

¹ See <python/typeshed#12117> (implicated)
  and <python/typeshed#12181> (open, may fix?)
  and <python/typeshed#12182> (open, may fix?)
@tsibley tsibley mentioned this issue Sep 5, 2024
Merged
1 task
@kbaikov kbaikov mentioned this issue Nov 26, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix todo in tarfile for modes with pipe char kbaikov/typeshed
1 participant
@srittau