Upload async file data

[aersam]

Is your feature request related to a problem? Please describe.
Hi there! I'm trying to upload a block blob using an async source:

async with aiofiles.open(file=path, mode="rb") as data:
    await blob_client.upload_blob(data, metadata=tags, overwrite=True)

This doens't work, the data is supposed to be bytes.

Describe the solution you'd like
I'd like to be able to use an async stream as source for the async Blob Client

Describe alternatives you've considered
My current workaround is to load the whole file into RAM:

async with aiofiles.open(file=path, mode="rb") as data:
    dt = await data.read()
    await blob_client.upload_blob(dt, metadata=tags, overwrite=True)

As long as my files don't get too large, this will work

Additional context
Add any other context or screenshots about the feature request here.

[ghost]

Thank you for your feedback. This has been routed to the support team for assistance.

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.

Issue Details

---

Is your feature request related to a problem? Please describe.
Hi there! I'm trying to upload a block blob using an async source:

async with aiofiles.open(file=path, mode="rb") as data:
    await blob_client.upload_blob(data, metadata=tags, overwrite=True)

This doens't work, the data is supposed to be bytes.

Describe the solution you'd like
I'd like to be able to use an async stream as source for the async Blob Client

Describe alternatives you've considered
My current workaround is to load the whole file into RAM:

async with aiofiles.open(file=path, mode="rb") as data:
    dt = await data.read()
    await blob_client.upload_blob(dt, metadata=tags, overwrite=True)

As long as my files don't get too large, this will work

Additional context
Add any other context or screenshots about the feature request here.

Author:	aersam
Assignees:	jalauzon-msft, vincenttran-msft
Labels:	Storage, question, Service Attention, Client, customer-reported, needs-team-attention
Milestone:	-

[navba-MSFT]

@jalauzon-msft @vincenttran-msft Could you please look into this once you get a chance ? Thanks in advance.

[jalauzon-msft]

Hi @aersam Adrian, thanks for reaching out. Could you please share which version of the Blob SDK you are using? We recently released support for streams with an async read method in version 12.14.0 which I believe should work with aiofiles. Another thing to check is to make sure you are using an async BlobClient which would be imported from the aio module. Async streams are only supported on the async clients.

from azure.storage.blob.aio import BlobClient

[aersam]

Hi there
I'm using aiofiles 22.1 and azure-storage-blob 12.14.1

I am using the aio variants. who does not ? :)

pyright already complains when doing so:

    async with aiofiles.open(file=fake_path or data.path, mode="rb") as fdata:
        await blob_client.upload_blob(fdata, metadata=data.tags, overwrite=True)

the error at runtime:

  File "c:\Projects\DataHub\BlobTagging\blobtagging\main.py", line 96, in do
    await upload_blob(client, data=data, target_root=target_root, fake_path=fake_path)
  File "c:\Projects\DataHub\BlobTagging\blobtagging\main.py", line 90, in upload_blob 
    await blob_client.upload_blob(fdata, metadata=data.tags, overwrite=True)
  File "c:\Projects\DataHub\BlobTagging\.venv\lib\site-packages\azure\core\tracing\decorator_async.py", line 79, in wrapper_use_tracer
    return await func(*args, **kwargs)
  File "c:\Projects\DataHub\BlobTagging\.venv\lib\site-packages\azure\storage\blob\aio\_blob_client_async.py", line 405, in upload_blob
    return await upload_block_blob(**options)
  File "c:\Projects\DataHub\BlobTagging\.venv\lib\site-packages\azure\storage\blob\aio\_upload_helpers.py", line 79, in upload_block_blob        
    raise TypeError('Blob data should be of type bytes.')
TypeError: Blob data should be of type bytes.

[aersam]

it's a bit strange: I looked at your helper code and it basically does a check with asyncio.iscoroutinefunction(data.read) which I'd expect to return True, but it returns False. I'm not too deep into this python async stuff; I don't understnd why. My Python version is 3.9
Saw also python/cpython#81371

I could fix it, see my PR

[jalauzon-msft]

Hi @aersam Adrian, thanks for the PR! Strange that aiofiles async read method does not work with iscoroutinefunction but your proposed change does seem like a reasonable workaround. Let me check with some other folks on my team, that are more familiar with this async Python stuff, next week to make sure the change will work is all other scenarios.

If they approve, there are one or two other places we'd need to apply this change at the same time that I may ask you to do or just submit a PR myself if I have the time.

[jalauzon-msft]

Hi again @aersam Adrian, I discussed with the team, and we think your change looks good and should help to increase our compatibility with different stream types. There were a few more places we needed to make this change in addition to your PR, so I went ahead and created #28472 instead of making you track them down. I will close #28410 in favor of this one.