[coroutines/c++] Return object is being prematurely converted.

[no-more-secrets]

using llvm/clang main built on 2022-07-14.

There appears to be a regression in how coroutine return objects are implicitly converted prior to returning them.

First as background, the get_return_object customization point is allowed to return an object that can be implicitly converted to the real return type. This part is still working fine.

The regression concerns when this implicit conversion happens.

Clang traditionally waited to perform this implicit conversion until after the return_value customization point was called. This allowed doing some tricks that are useful in implementing e.g. a std::optional coroutine. So if you were to print out the order of calls, it would looks like this traditionally:

1. call get_return_object
2. call return_value
3. do implicit conversion of return object to the type that is actually needed.

This may have changed at some point in the last few months, but the version of clang that I built today does it in a different order:

1. call get_return object
2. do implicit conversion of return object to the type that is actually needed.
3. call return_value

I'm not sure if the standard guarantees that it will happen in a certain order... but I know that traditionally it used to be done according to the first ordering (which gcc also does), but now that changed, and it is breaking my implementation of the std::optional coroutine.

It seems sensible to wait as long as possible to perform the implicit conversion, since it affords flexibility to the user in that it permits the object that get_return_object returns to remain alive and accessible to return_value (or other customization point methods) until the very end when it is implicitly converted just before ending the coroutine.

[no-more-secrets]

If someone needs a minimal reproducer I can provide one, but I'm hoping that it won't be necessary.

[yuanfang-chen]

@ChuanqiXu9 maybe this relates to https://reviews.llvm.org/D117087?

[ChuanqiXu9]

@ChuanqiXu9 maybe this relates to https://reviews.llvm.org/D117087?

I guess so. But I am not sure if it is incorrect.

@dpacbach I don't understand how this is related to return_value. From my point of view, get_return_object is not related with return_value. A reduced example may be helpful to understand. Thanks

[ChuanqiXu9]

@tstellar Hi, may you love to create a new label like coroutines? Recently the number of coroutine issues goes high and it is relatively independent with C++20. So it would be helpful to create a new issue label.

[EugeneZelenko]

@ChuanqiXu9: Done, but you could add labels yourself.

[no-more-secrets]

@ChuanqiXu9 Here is a demonstration of the change: https://godbolt.org/z/jfrqsKsnb

There you can see clang trunk vs. clang 14 vs gcc 12.1. Clang 14 and gcc 12.1 agree in that they perform the implicit conversion after the final_suspend (which is even later than I thought, but that's fine). However, clang trunk does the implicit conversion just after calling get_return_object, which I think could be considered a regression. Note that MSVC also agrees with clang 14 and gcc (https://godbolt.org/z/Mcbe8Pd7x), so clang trunk is the only one that does this new behavior.

Note that on the cppreference coroutines page it says this:

When a coroutine reaches a suspension point: the return object obtained earlier is returned to the caller/resumer, after implicit conversion to the return type of the coroutine, if necessary.

To me that implies that it should do the implicit conversion only when necessary (as late as possible), though I am not sure what the C++ standard says officially. At the very least it seems a bit suspicious that clang trunk disagrees with the other two major compilers.

[no-more-secrets]

@ChuanqiXu9 Here is another library that will be broken by this change, as it relies on the deferred implicit conversion of the get_return_object, as stated in its readme:

https://github.com/toby-allsopp/coroutine_monad

Requirements
The implementation requires ... Clang 5 or later...

In particular, the MSVC implementation converts the object returned by get_return_object to the return type of the coroutine immediately rather than waiting until the coroutine returns to its caller.

[no-more-secrets]

Here is another issue from Lewis Baker (who I'd think is a reliable source on this) who seems to imply that the conversion should be deferred:

lewissbaker/cppcoro#43

[ChuanqiXu9]

I got it. It surprises me but it looks like the intentional behavior indeed. I just failed to find the corresponding wording. I would try to fix it after I understand how to handle it properly.

[ChuanqiXu9]

@dpacbach from https://cplusplus.github.io/CWG/issues/2563.html, it looks like the current behavior of clang is fine.