asyncio.wait_for: process future result produced during cancelation

[romasku]

BPO	40672
Nosy	@asvetlov, @cjerdonek, @1st1, @romasku

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-05-18.17:23:20.075>
labels = ['type-feature', '3.9', 'expert-asyncio']
title = 'asyncio.wait_for: process future result produced during cancelation'
updated_at = <Date 2020-05-18.22:51:20.953>
user = 'https://github.com/romasku'

bugs.python.org fields:

activity = <Date 2020-05-18.22:51:20.953>
actor = 'chris.jerdonek'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2020-05-18.17:23:20.075>
creator = 'Roman Skurikhin'
dependencies = []
files = []
hgrepos = []
issue_num = 40672
keywords = []
message_count = 3.0
messages = ['369278', '369285', '369308']
nosy_count = 4.0
nosy_names = ['asvetlov', 'chris.jerdonek', 'yselivanov', 'Roman Skurikhin']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40672'
versions = ['Python 3.9']

[romasku]

In https://bugs.python.org/issue40607 asyncio.wait_for behavior was changed so it propagates exceptions that happened during cancellation. But it still raises TimeoutError if cancelation ends with some value being returned. In the following example value 42 is lost:

import asyncio


async def return_42_on_cancel():
    try:
        await asyncio.sleep(20)
    except asyncio.CancelledError:
        return 42  # `return` is useless in this block.


async def main():
    try:
        await asyncio.wait_for(return_42_on_cancel(), timeout=1)
    except asyncio.TimeoutError:
        print('Timeout')

asyncio.run(main())

I think it's better to either:

1. Return that value from asyncio.wait_for.

The motivation here is that if the task returns something, we shouldn't conceal it. I also searched through GitHub and found some places where others catch CancelledError and return value (https://github.com/grpc/grpc/blob/44fb37c99f2853cc23f04fba15468980d9e28e41/src/python/grpcio/grpc/experimental/aio/_interceptor.py#L328).

It can also be used with some coroutines developed to be wrapped with wait_for, for example suppose the following equation solving function:

async def solve_iteratively(initial_x, next_approximation):
    result = initial_x
    try:
        while True:
            result = next_approximation(result)
            await asyncio.sleep(0)
    except asyncio.CancelledError:
        return result

It allows us to control its execution time using asyncio.wait_for.

2. Add some warning about the value is thrown away (in debug mode) and document it somewhere.

===

I am a newbie here, so sorry if it is wrong to create such "proposal" issues.

[1st1]

2. Add some warning about the value is thrown away (in debug mode) and document it somewhere.

The documentation update is definitely something that needs to be done in 3.9. Want to submit a PR?

We can also issue a warning in asyncio debug mode.

I'm really not sure about "1) Return that value from asyncio.wait_for.".

I am a newbie here, so sorry if it is wrong to create such "proposal" issues.

The issue is clear, thanks for submitting it! Keep up the great work.

[cjerdonek]

Regarding the documentation, I'm not sure we _need_ to say what happens in this edge case for 3.9. It was already unspecified before 3.9, so we're not any worse off. (The change in bpo-40607 was, however, documented.)

I'd rather come to agreement on (1) first. Because if we document it now, then it could be interpreted as defined behavior and so harder to change later due to backwards compat guarantees.

[kumaraditya303]

Duplicate of #86296

[gvanrossum]

Duplicate of #86296

[gvanrossum]

(And FWIW I think there we're planning to propagate CancelledError in that case.)

@romasku