Skip to content

gh-82500: Fix asyncio sendfile overflow on 32bit #107056

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 23, 2023

Conversation

bdraco
Copy link
Contributor

@bdraco bdraco commented Jul 22, 2023

asyncio has the same issue

This is the same patch we have been running for a while with Home Assistant https://raw.githubusercontent.com/home-assistant/docker-base/master/python/3.11/asynctio_unix_events.patch

related issue and trace home-assistant/supervisor#4375

File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 458, in _handle_request
   reset = await self.finish_response(request, resp, start_time)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 611, in finish_response
   await prepare_meth(request)
 File "/usr/local/lib/python3.11/site-packages/aiohttp/web_fileresponse.py", line 286, in prepare
   return await self._sendfile(request, fobj, offset, count)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/aiohttp/web_fileresponse.py", line 99, in _sendfile
   await loop.sendfile(transport, fobj, offset, count)
 File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1185, in sendfile
   return await self._sendfile_native(transport, file,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 725, in _sendfile_native
   return await self.sock_sendfile(transp._sock, file, offset, count,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/asyncio/base_events.py", line 881, in sock_sendfile
   return await self._sock_sendfile_native(sock, file,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/asyncio/unix_events.py", line 373, in _sock_sendfile_native
   return await fut
          ^^^^^^^^^
 File "/usr/local/lib/python3.11/asyncio/unix_events.py", line 395, in _sock_sendfile_native_impl
   sent = os.sendfile(fd, fileno, offset, blocksize)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: Python int too large to convert to C ssize_t

@bdraco bdraco marked this pull request as ready for review July 22, 2023 21:18
Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is definitely worth fixing. I have a few nits.

Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@bdraco
Copy link
Contributor Author

bdraco commented Jul 23, 2023

thanks

@bdraco bdraco deleted the sendfile_overflow_32bit branch July 23, 2023 04:10
jtcave pushed a commit to jtcave/cpython that referenced this pull request Jul 23, 2023
mementum pushed a commit to mementum/cpython that referenced this pull request Jul 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants