Skip to content

Commit 755b385

Browse files
committed
gh-129005: Fix buffer expansion in _pyio.FileIO.readall
Move to a linear slice append with an iterator which has a length hint. This is more expensive then PyByteArray_Resize, but I think as efficient as can get without a new bytearray Python API to resize. The previous code didn't append as I had intended: ```python a = bytearray() >>> a[0:5] = b'\0' >>> a bytearray(b'\x00') >>> a[5:16] = b'\01' >>> a bytearray(b'\x00\x01') >>> len(a) 2 ```
1 parent d89a5f6 commit 755b385

File tree

2 files changed

+4
-1
lines changed

2 files changed

+4
-1
lines changed

Lib/_pyio.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
import sys
1111
# Import _thread instead of threading to reduce startup cost
1212
from _thread import allocate_lock as Lock
13+
from itertools import repeat
1314
if sys.platform in {'win32', 'cygwin'}:
1415
from msvcrt import setmode as _setmode
1516
else:
@@ -1686,7 +1687,8 @@ def readall(self):
16861687
if addend < DEFAULT_BUFFER_SIZE:
16871688
addend = DEFAULT_BUFFER_SIZE
16881689
bufsize += addend
1689-
result[bytes_read:bufsize] = b'\0'
1690+
result.extend(repeat(0, addend))
1691+
assert len(result) == bufsize, "Should have expanded in size"
16901692
assert bufsize - bytes_read > 0, "Should always try and read at least one byte"
16911693
try:
16921694
n = os.readinto(self._fd, memoryview(result)[bytes_read:])
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
:mod:`!pyio`: Fix expansion of buffer in _pyio.FileIO.readall

0 commit comments

Comments
 (0)