Skip to content

Commit c7d8931

Browse files
committed
Use memoryviews for writes only for Python 3.10+
The core SDK validates that if a bytes iterable is provided that the object has an __iter__() method available: https://github.com/Azure/azure-sdk-for-python/blob/08e0cd870062e11e22998d6e10ef55581b35c045/sdk/core/azure-core/azure/core/rest/_helpers.py#L146 However memoryview objects did not expose an __iter__() until Python 3.10 via: python/cpython#22119 We still want to leverage memoryviews when possible to reduce the number of potential copies so we wrap the bytearrays sent for stage_blocks if we are on Python 3.10+. Otherwise, we just directly provide the bytearray as is.
1 parent 8b7275b commit c7d8931

File tree

1 file changed

+15
-0
lines changed

1 file changed

+15
-0
lines changed

src/azstoragetorch/_client.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
import math
1212
import os
1313
import random
14+
import sys
1415
import threading
1516
import time
1617
import urllib.parse
@@ -197,6 +198,11 @@ def stage_blocks(
197198
) -> List[STAGE_BLOCK_FUTURE_TYPE]:
198199
if not data:
199200
raise ValueError("Data must not be empty.")
201+
if (
202+
isinstance(data, memoryview)
203+
and not self._sdk_supports_memoryview_for_writes()
204+
):
205+
data = data.obj
200206
stage_block_partitions = self._get_stage_block_partitions(data)
201207
futures = []
202208
for pos, length in stage_block_partitions:
@@ -371,3 +377,12 @@ def _get_url_without_query_string(
371377
None,
372378
)
373379
)
380+
381+
def _sdk_supports_memoryview_for_writes(self) -> bool:
382+
# The SDK validates objects passed to its HTTP request layer expose an __iter__()
383+
# method that can be used to iterate through bytes passed to it. However, memoryview
384+
# objects did not expose an __iter__() method till Python 3.10.
385+
#
386+
# We still want to leverage memorviews when we can to avoid unnecessary copies. So
387+
# we check the Python version to determine if we can use memoryviews for writes.
388+
return sys.version_info >= (3, 10)

0 commit comments

Comments
 (0)