[C++20] [Coroutines] Don't mark await_suspend as noinline if it is specified as always_inline already

Address #64933 and partially
#64945.

After c467245, we will add a noinline attribute to the await_suspend
member function of an awaiter if the awaiter has any non static member
functions.

Obviously, this decision will bring some performance regressions. And
people may complain about this while the long term solution may not be
available soon. In such cases, it is better to provide a solution for
the users who met the regression surprisingly.

Also it is natural to not prevent the inlining if the function is marked
as always_inline by the users already.

Author: ChuanqiXu9

clang/lib/CodeGen/CGCall.cpp

@@ -5548,7 +5548,11 @@ RValue CodeGenFunction::EmitCall(const CGFunctionInfo &CallInfo,
   // Then it is much easier to perform the safety analysis in the middle end.
   // If it is safe to inline the call to awaitSuspend, we can replace it in the
   // CoroEarly pass. Otherwise we could replace it in the CoroSplit pass.
-  if (inSuspendBlock() && mayCoroHandleEscape())
+  //
+  // If the `await_suspend()` function is marked as `always_inline` explicitly,
+  // we should give the user the right to control the codegen.
+  if (inSuspendBlock() && mayCoroHandleEscape() &&
+      !TargetDecl->hasAttr<AlwaysInlineAttr>())
     Attrs = Attrs.addFnAttribute(getLLVMContext(), llvm::Attribute::NoInline);
 
   // Disable inlining inside SEH __try blocks.

clang/test/CodeGenCoroutines/coro-awaiter-noinline-suspend.cpp

@@ -67,6 +67,17 @@ StatefulAwaiter& operator co_await(Awaitable2) {
   return GlobalAwaiter;
 }
 
+struct AlwaysInlineStatefulAwaiter {
+    void* value;
+    bool await_ready() const noexcept { return false; }
+
+    template <typename PromiseType>
+    __attribute__((always_inline))
+    void await_suspend(std::coroutine_handle<PromiseType> h) noexcept {}
+
+    void await_resume() noexcept {}
+};
+
 Task testing() {
     co_await std::suspend_always{};
     co_await StatefulAwaiter{};
@@ -85,6 +96,8 @@ Task testing() {
 
     co_await Awaitable{};
     co_await Awaitable2{};
+
+    co_await AlwaysInlineStatefulAwaiter{};
 }
 
 // CHECK-LABEL: @_Z7testingv
@@ -135,6 +148,10 @@ Task testing() {
 // CHECK: call token @llvm.coro.save
 // CHECK: call void @_ZN15StatefulAwaiter13await_suspendIN4Task12promise_typeEEEvSt16coroutine_handleIT_E{{.*}}#[[NOINLINE_ATTR]]
 
+// Check `co_await AlwaysInlineStatefulAwaiter{};` to make sure user can force the await_suspend function to get inlined.
+// CHECK: call token @llvm.coro.save
+// CHECK: call void @_ZN27AlwaysInlineStatefulAwaiter13await_suspendIN4Task12promise_typeEEEvSt16coroutine_handleIT_E{{.*}}#[[NORMAL_ATTR]]
+
 // Check `co_await __promise__.final_suspend();`. We don't emit an blocker here since it is
 // empty.
 // CHECK: call token @llvm.coro.save