Skip to content

partial_apply support for coroutines #71653

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Aug 9, 2024
Merged

Conversation

asavonic
Copy link
Contributor

@asavonic asavonic commented Feb 15, 2024

The patch adds lowering of partial_apply instructions for coroutines.
This pattern seems to trigger a lot of type mismatch errors in IRGen, because coroutine functions are not substituted in the same way as regular functions (see the patch 07f03bd "Use pattern substitutions to consistently abstract yields" for more details). The odd type conversions in the patch are related to this issue, and these should be checked carefully. Perhaps it is better to enable substitutions for coroutine functions instead (at least for some cases).

Other than that, lowering of partial_apply for coroutines is straightforward: we generate another coroutine that captures arguments passed to the partial_apply instructions. It calls the original coroutine for yields (first return) and yields the resulting values. Then it calls the original function's continuation for return or unwind, and forwards them to the caller as well.

After IRGen, LLVM's Coroutine pass transforms the generated coroutine (along with all other coroutines) and eliminates llvm.coro.* intrinsics. LIT tests check LLVM IR after this transformation.

Test cases are taken from partial_apply.sil test, and adapted to use coroutines.

@asavonic asavonic marked this pull request as draft February 15, 2024 16:24
@asl
Copy link
Contributor

asl commented Feb 15, 2024

@rjmccall Will you please take a look on fba0135 ?

@asavonic asavonic force-pushed the yield-diff-pa branch 2 times, most recently from 66c56f6 to 8f46ff7 Compare February 16, 2024 17:59
@asl
Copy link
Contributor

asl commented Feb 16, 2024

@swift-ci please test

@asl
Copy link
Contributor

asl commented Feb 16, 2024

Please test with swiftlang/llvm-project#8199
@swift-ci please test

@asl
Copy link
Contributor

asl commented Feb 19, 2024

@swift-ci please test

@asl
Copy link
Contributor

asl commented Feb 21, 2024

Please test with swiftlang/llvm-project#8199
@swift-ci please test

@asl asl force-pushed the yield-diff-pa branch from 9e805a2 to 1ed6d45 Compare April 5, 2024 19:20
@asl
Copy link
Contributor

asl commented Apr 5, 2024

@swift-ci please test

@asavonic
Copy link
Contributor Author

A few things have changed since Feb:

  1. Copyable and Escapable seem to be implicitly added to type specifiers. We need to add them to input SIL.
  2. The workaround with class F now causes a crash after bitcast from float to ptr. I think this workaround was incorrect in the first place, because we substitute F with unrelated type Float.
  3. There is a new crash with a cast to LoadableTypeInfo. The current patch already handles some of these cases (check isa<LoadableTypeInfo>), when a generic type (they are not loadable) appears where it is not expected. Unfortunately, I was not able to debug this further.

Here is a WIP patchset that partially addresses (1) and (2), and triggers (3):
https://github.com/asavonic/swift/tree/yield-diff-pa-regression

@asl asl force-pushed the yield-diff-pa branch from 1ed6d45 to 29e830a Compare May 17, 2024 05:01
@asl
Copy link
Contributor

asl commented May 17, 2024

@swift-ci please test

@asl asl force-pushed the yield-diff-pa branch from 29e830a to 21934fd Compare May 17, 2024 16:24
@asl
Copy link
Contributor

asl commented May 17, 2024

@swift-ci please test linux

@asl asl force-pushed the yield-diff-pa branch from 21934fd to 82d1bb8 Compare May 17, 2024 17:55
@asl
Copy link
Contributor

asl commented May 17, 2024

@swift-ci please test linux

@asl
Copy link
Contributor

asl commented May 17, 2024

@swift-ci please test windows

@asl
Copy link
Contributor

asl commented May 17, 2024

@swift-ci please test macos

@asl asl force-pushed the yield-diff-pa branch from 82d1bb8 to e756fe8 Compare May 23, 2024 01:36
@asl
Copy link
Contributor

asl commented May 23, 2024

@swift-ci please test

@asl
Copy link
Contributor

asl commented Aug 5, 2024

@aschwaighofer @rjmccall Will you please take a look into this patch? It was reworked and now I believe much cleaner.

In addition, all tests from partial_apply.sil (that tests partial apply of normal functions) were ported to coroutines and therefore we believe that the codegen of partial_apply of a coroutine is on par with the one of normal functions, so there are no TODOs, FIXMEs, etc.

So, to conclude: there was no support for partial_apply of a coroutine before and this PR closes this implementation gap.

Thanks!

Copy link
Contributor

@eeckstein eeckstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a paragraph about partial-apply co-routines into docs/SIL.rst (under the section for partial_apply)? Like you wrote in the description of this PR. Please also add that this is (currently) only used for auto-diff.

Do escaping and non-escaping ([on_stack]) partial_applys support co-routines? If not, please document this as well (and add a check in the SIL verifier).

@asl
Copy link
Contributor

asl commented Aug 5, 2024

Do escaping and non-escaping ([on_stack]) partial_applys support co-routines? If not, please document this as well (and add a check in the SIL verifier).

@eeckstein This is one of testcases (see https://github.com/swiftlang/swift/pull/71653/files#diff-2eac67a6f96297f40450a009472fdbb4cdb09445c4eb3f9a8b69550e19f2b9b3R600 and below)

sil @partial_apply_class_on_stack : $@convention(thin) (@owned SwiftClass) -> () {
entry(%a : $SwiftClass):
  %f = function_ref @partially_applyable_to_class : $@yield_once @convention(thin) (@guaranteed SwiftClass) -> (@yields SwiftClass)
  %c = partial_apply [callee_guaranteed] [on_stack] %f(%a) : $@yield_once @convention(thin) (@guaranteed SwiftClass) -> (@yields SwiftClass)
  %use = function_ref @use_closure : $@convention(thin) (@noescape @yield_once @callee_guaranteed () -> (@yields SwiftClass)) -> ()
  apply %use(%c) : $@convention(thin) (@noescape @yield_once @callee_guaranteed () -> (@yields SwiftClass)) -> ()
  dealloc_stack %c : $@noescape @yield_once @callee_guaranteed () -> (@yields SwiftClass)
  strong_release %a : $SwiftClass
  %t = tuple()
  return %t : $()
}

So it seems to be there

@asl asl requested a review from jckarter as a code owner August 7, 2024 05:12
@asl
Copy link
Contributor

asl commented Aug 7, 2024

@swift-ci please test

@asl
Copy link
Contributor

asl commented Aug 7, 2024

@eeckstein I added the note to docs in 962777f

Please let me know if this is enough or something else should be added

@asl
Copy link
Contributor

asl commented Aug 7, 2024

preset=buildbot,tools=RA,stdlib=RD,test=non_executable
@swift-ci Please test with preset macOS

@eeckstein
Copy link
Contributor

I added the note to docs in 962777f

Perfect, thanks!

@asl
Copy link
Contributor

asl commented Aug 8, 2024

@swift-ci please test

@asl
Copy link
Contributor

asl commented Aug 8, 2024

preset=buildbot,tools=RA,stdlib=RD,test=non_executable
@swift-ci Please test with preset macOS

Copy link
Contributor

@aschwaighofer aschwaighofer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

asavonic and others added 12 commits August 8, 2024 08:53
The patch adds lowering of partial_apply instructions for coroutines.

This pattern seems to trigger a lot of type mismatch errors in IRGen, because
coroutine functions are not substituted in the same way as regular functions
(see the patch 07f03bd "Use pattern substitutions to consistently abstract
yields" for more details). The odd type conversions in the patch are related to
this issue, and these should be checked carefully. Perhaps it is better to
enable substitutions for coroutine functions instead (at least for some cases).

Other than that, lowering of partial_apply for coroutines is straightforward: we
generate another coroutine that captures arguments passed to the partial_apply
instructions. It calls the original coroutine for yields (first return) and
yields the resulting values. Then it calls the original function's continuation
for return or unwind, and forwards them to the caller as well.

After IRGen, LLVM's Coroutine pass transforms the generated coroutine (along with
all other coroutines) and eliminates llvm.coro.* intrinsics. LIT tests check
LLVM IR after this transformation.
A placeholder goes instead of the context in case of error results, so
that thin functions have the same number of arguments as thick
ones. Coroutine lowering adds it as part of getCallEmission (via
setFromCallee), so we don't need to do it in
emitPartialApplicationForwarder.
@asl
Copy link
Contributor

asl commented Aug 8, 2024

@swift-ci please test

@asl
Copy link
Contributor

asl commented Aug 8, 2024

preset=buildbot,tools=RA,stdlib=RD,test=non_executable
@swift-ci Please test with preset macOS

@asl
Copy link
Contributor

asl commented Aug 8, 2024

Buildbot configiration fails due Parse/enum_floating_point_raw_value.swift test broken on non-x86 targets after 6610439 and is unrelated to this PR

@asl asl merged commit 5aa9d3e into swiftlang:main Aug 9, 2024
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants