Skip to content

[C++20] [Modules] Merge lambdas in source to imported lambdas #106483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 29, 2024

Conversation

ChuanqiXu9
Copy link
Member

Close #102721

Generally, the type of merged decls will be reused in ASTContext. But for lambda, in the import and then include case, we can't decide its previous decl in the imported modules so that we can't assign the previous decl before creating the type for it. Since we can't decide its numbering before creating it. So we have to assign the previous decl and the canonical type for it after creating it, which is unusual and slightly hack.

@ChuanqiXu9 ChuanqiXu9 added clang:modules C++20 modules and Clang Header Modules skip-precommit-approval PR for CI feedback, not intended for review labels Aug 29, 2024
@ChuanqiXu9 ChuanqiXu9 self-assigned this Aug 29, 2024
@llvmbot llvmbot added clang Clang issues not falling into any other category clang:frontend Language frontend issues, e.g. anything involving "Sema" labels Aug 29, 2024
@llvmbot
Copy link
Member

llvmbot commented Aug 29, 2024

@llvm/pr-subscribers-clang

@llvm/pr-subscribers-clang-modules

Author: Chuanqi Xu (ChuanqiXu9)

Changes

Close #102721

Generally, the type of merged decls will be reused in ASTContext. But for lambda, in the import and then include case, we can't decide its previous decl in the imported modules so that we can't assign the previous decl before creating the type for it. Since we can't decide its numbering before creating it. So we have to assign the previous decl and the canonical type for it after creating it, which is unusual and slightly hack.


Full diff: https://github.com/llvm/llvm-project/pull/106483.diff

8 Files Affected:

  • (modified) clang/include/clang/AST/Decl.h (+1)
  • (modified) clang/include/clang/AST/Type.h (+1)
  • (modified) clang/include/clang/Sema/ExternalSemaSource.h (+1-1)
  • (modified) clang/include/clang/Sema/MultiplexExternalSemaSource.h (+1-1)
  • (modified) clang/include/clang/Serialization/ASTReader.h (+1-1)
  • (modified) clang/lib/Sema/MultiplexExternalSemaSource.cpp (+1-1)
  • (modified) clang/lib/Serialization/ASTReader.cpp (+27-8)
  • (added) clang/test/Modules/pr102721.cppm (+33)
diff --git a/clang/include/clang/AST/Decl.h b/clang/include/clang/AST/Decl.h
index 6d84bd03de810a..0600ecc4d14a18 100644
--- a/clang/include/clang/AST/Decl.h
+++ b/clang/include/clang/AST/Decl.h
@@ -3366,6 +3366,7 @@ class IndirectFieldDecl : public ValueDecl,
 /// Represents a declaration of a type.
 class TypeDecl : public NamedDecl {
   friend class ASTContext;
+  friend class ASTReader;
 
   /// This indicates the Type object that represents
   /// this TypeDecl.  It is a cache maintained by
diff --git a/clang/include/clang/AST/Type.h b/clang/include/clang/AST/Type.h
index 575f3c17a3f691..50041915204a1f 100644
--- a/clang/include/clang/AST/Type.h
+++ b/clang/include/clang/AST/Type.h
@@ -1684,6 +1684,7 @@ class ExtQualsTypeCommonBase {
   friend class ExtQuals;
   friend class QualType;
   friend class Type;
+  friend class ASTReader;
 
   /// The "base" type of an extended qualifiers type (\c ExtQuals) or
   /// a self-referential pointer (for \c Type).
diff --git a/clang/include/clang/Sema/ExternalSemaSource.h b/clang/include/clang/Sema/ExternalSemaSource.h
index 22d1ee2df115a6..11cd69df88d1c1 100644
--- a/clang/include/clang/Sema/ExternalSemaSource.h
+++ b/clang/include/clang/Sema/ExternalSemaSource.h
@@ -233,7 +233,7 @@ class ExternalSemaSource : public ExternalASTSource {
   /// Notify the external source that a lambda was assigned a mangling number.
   /// This enables the external source to track the correspondence between
   /// lambdas and mangling numbers if necessary.
-  virtual void AssignedLambdaNumbering(const CXXRecordDecl *Lambda) {}
+  virtual void AssignedLambdaNumbering(CXXRecordDecl *Lambda) {}
 
   /// LLVM-style RTTI.
   /// \{
diff --git a/clang/include/clang/Sema/MultiplexExternalSemaSource.h b/clang/include/clang/Sema/MultiplexExternalSemaSource.h
index 238fb398b7d129..3d1906d8699265 100644
--- a/clang/include/clang/Sema/MultiplexExternalSemaSource.h
+++ b/clang/include/clang/Sema/MultiplexExternalSemaSource.h
@@ -361,7 +361,7 @@ class MultiplexExternalSemaSource : public ExternalSemaSource {
                                         QualType T) override;
 
   // Inform all attached sources that a mangling number was assigned.
-  void AssignedLambdaNumbering(const CXXRecordDecl *Lambda) override;
+  void AssignedLambdaNumbering(CXXRecordDecl *Lambda) override;
 
   /// LLVM-style RTTI.
   /// \{
diff --git a/clang/include/clang/Serialization/ASTReader.h b/clang/include/clang/Serialization/ASTReader.h
index 2d8952ddbd71df..898f4392465fdf 100644
--- a/clang/include/clang/Serialization/ASTReader.h
+++ b/clang/include/clang/Serialization/ASTReader.h
@@ -2149,7 +2149,7 @@ class ASTReader
       llvm::MapVector<const FunctionDecl *, std::unique_ptr<LateParsedTemplate>>
           &LPTMap) override;
 
-  void AssignedLambdaNumbering(const CXXRecordDecl *Lambda) override;
+  void AssignedLambdaNumbering(CXXRecordDecl *Lambda) override;
 
   /// Load a selector from disk, registering its ID if it exists.
   void LoadSelector(Selector Sel);
diff --git a/clang/lib/Sema/MultiplexExternalSemaSource.cpp b/clang/lib/Sema/MultiplexExternalSemaSource.cpp
index 79e656eb4b7e27..cd44483b5cbe04 100644
--- a/clang/lib/Sema/MultiplexExternalSemaSource.cpp
+++ b/clang/lib/Sema/MultiplexExternalSemaSource.cpp
@@ -343,7 +343,7 @@ bool MultiplexExternalSemaSource::MaybeDiagnoseMissingCompleteType(
 }
 
 void MultiplexExternalSemaSource::AssignedLambdaNumbering(
-    const CXXRecordDecl *Lambda) {
+    CXXRecordDecl *Lambda) {
   for (auto *Source : Sources)
     Source->AssignedLambdaNumbering(Lambda);
 }
diff --git a/clang/lib/Serialization/ASTReader.cpp b/clang/lib/Serialization/ASTReader.cpp
index ffdaec4067e1c4..a0abd4a2bd0f5b 100644
--- a/clang/lib/Serialization/ASTReader.cpp
+++ b/clang/lib/Serialization/ASTReader.cpp
@@ -8963,15 +8963,34 @@ void ASTReader::ReadLateParsedTemplates(
   LateParsedTemplates.clear();
 }
 
-void ASTReader::AssignedLambdaNumbering(const CXXRecordDecl *Lambda) {
-  if (Lambda->getLambdaContextDecl()) {
-    // Keep track of this lambda so it can be merged with another lambda that
-    // is loaded later.
-    LambdaDeclarationsForMerging.insert(
-        {{Lambda->getLambdaContextDecl()->getCanonicalDecl(),
-          Lambda->getLambdaIndexInContext()},
-         const_cast<CXXRecordDecl *>(Lambda)});
+void ASTReader::AssignedLambdaNumbering(CXXRecordDecl *Lambda) {
+  if (!Lambda->getLambdaContextDecl())
+    return;
+
+  auto LambdaInfo =
+      std::make_pair(Lambda->getLambdaContextDecl()->getCanonicalDecl(),
+                     Lambda->getLambdaIndexInContext());
+
+  // Handle the import and then include case for lambdas.
+  if (auto Iter = LambdaDeclarationsForMerging.find(LambdaInfo);
+      Iter != LambdaDeclarationsForMerging.end() &&
+      Iter->second->isFromASTFile() && Lambda->getFirstDecl() == Lambda) {
+    CXXRecordDecl *Previous =
+        cast<CXXRecordDecl>(Iter->second)->getMostRecentDecl();
+    Lambda->setPreviousDecl(Previous);
+    // FIXME: It will be best to use the Previous type when we creating the
+    // lambda directly. But that requires us to get the lambda context decl and
+    // lambda index before creating the lambda, which needs a drastic change in
+    // the parser.
+    const_cast<QualType &>(Lambda->TypeForDecl->CanonicalType) =
+        Previous->TypeForDecl->CanonicalType;
+    return;
   }
+
+  // Keep track of this lambda so it can be merged with another lambda that
+  // is loaded later.
+  LambdaDeclarationsForMerging.insert(
+      {LambdaInfo, const_cast<CXXRecordDecl *>(Lambda)});
 }
 
 void ASTReader::LoadSelector(Selector Sel) {
diff --git a/clang/test/Modules/pr102721.cppm b/clang/test/Modules/pr102721.cppm
new file mode 100644
index 00000000000000..6a84393bcbd1f6
--- /dev/null
+++ b/clang/test/Modules/pr102721.cppm
@@ -0,0 +1,33 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+// RUN: split-file %s %t
+//
+// RUN: %clang_cc1 -std=c++20 %t/a.cppm -emit-module-interface -o %t/a.pcm
+// RUN: %clang_cc1 -std=c++20 %t/b.cppm -emit-module-interface -o %t/b.pcm \
+// RUN:   -fprebuilt-module-path=%t
+// RUN: %clang_cc1 -std=c++20 %t/test.cc -fsyntax-only -verify \
+// RUN:   -fprebuilt-module-path=%t
+
+//--- foo.h
+inline auto x = []{};
+
+//--- a.cppm
+module;
+#include "foo.h"
+export module a;
+export using ::x;
+
+//--- b.cppm
+module;
+import a;
+#include "foo.h"
+export module b;
+export using ::x;
+
+//--- test.cc
+// expected-no-diagnostics
+import a;
+import b;
+void test() {
+  x();
+}

@ChuanqiXu9 ChuanqiXu9 merged commit 55cdb3c into llvm:main Aug 29, 2024
13 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Aug 29, 2024

LLVM Buildbot has detected a new failure on builder sanitizer-x86_64-linux running on sanitizer-buildbot2 while building clang at step 2 "annotate".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/66/builds/3197

Here is the relevant piece of the build log for the reference
Step 2 (annotate) failure: 'python ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py' (failure)
...
[379/384] Generating MSAN_INST_GTEST.gtest-all.cc.x86_64.o
[380/384] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.x86_64-with-call.o
[381/384] Generating Msan-x86_64-with-call-Test
[382/384] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.x86_64.o
[383/384] Generating Msan-x86_64-Test
[383/384] Running compiler_rt regression tests
llvm-lit: /home/b/sanitizer-x86_64-linux/build/llvm-project/llvm/utils/lit/lit/main.py:72: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds.
-- Testing: 10229 tests, 88 workers --
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
TIMEOUT: SanitizerCommon-hwasan-x86_64-Linux :: Posix/fork_threaded.c (10229 of 10229)
******************** TEST 'SanitizerCommon-hwasan-x86_64-Linux :: Posix/fork_threaded.c' FAILED ********************
Exit Code: -9
Timeout: Reached timeout of 900 seconds

Command Output (stderr):
--
RUN: at line 1: /home/b/sanitizer-x86_64-linux/build/build_default/./bin/clang  -gline-tables-only -fsanitize=hwaddress -fuse-ld=lld -fsanitize-hwaddress-experimental-aliasing  -m64 -funwind-tables  -I/home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test -ldl -O0 /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/sanitizer_common/TestCases/Posix/fork_threaded.c -o /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp && env HWASAN_OPTIONS=die_after_fork=0  /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp
+ /home/b/sanitizer-x86_64-linux/build/build_default/./bin/clang -gline-tables-only -fsanitize=hwaddress -fuse-ld=lld -fsanitize-hwaddress-experimental-aliasing -m64 -funwind-tables -I/home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test -ldl -O0 /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/sanitizer_common/TestCases/Posix/fork_threaded.c -o /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp
+ env HWASAN_OPTIONS=die_after_fork=0 /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp

=================================================================
==2129157==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 568 byte(s) in 3 object(s) allocated from:
    #0 0x5cb2514093c4 in malloc /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/lib/hwasan/hwasan_allocation_functions.cpp:151:3
    #1 0x5cb2515621fe in operator new(unsigned long) llvm-link
    #2 0x5cb2513ff9b4 in _start (/home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp+0xef9b4)

Direct leak of 204 byte(s) in 1 object(s) allocated from:
    #0 0x5cb251408f49 in calloc /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/lib/hwasan/hwasan_allocation_functions.cpp:127:3
    #1 0x5cb2514ea72f in llvm::safe_calloc(unsigned long, unsigned long) llvm-link
    #2 0x5cb2513ff9b4 in _start (/home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp+0xef9b4)

Indirect leak of 1792 byte(s) in 43 object(s) allocated from:
    #0 0x5cb251408995 in aligned_alloc /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/lib/hwasan/hwasan_allocation_functions.cpp:58:3
    #1 0x5cb2514f8019 in llvm::allocate_buffer(unsigned long, unsigned long) llvm-link

Indirect leak of 780 byte(s) in 1 object(s) allocated from:
    #0 0x5cb251408f49 in calloc /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/lib/hwasan/hwasan_allocation_functions.cpp:127:3
    #1 0x5cb2514ea72f in llvm::safe_calloc(unsigned long, unsigned long) llvm-link

Indirect leak of 160 byte(s) in 1 object(s) allocated from:
    #0 0x5cb2514093c4 in malloc /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/lib/hwasan/hwasan_allocation_functions.cpp:151:3
    #1 0x5cb2515621fe in operator new(unsigned long) llvm-link
    #2 0x5cb2513ff9b4 in _start (/home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp+0xef9b4)

SUMMARY: HWAddressSanitizer: 3504 byte(s) leaked in 49 allocation(s).
==2129157==ERROR: HWAddressSanitizer: tag-mismatch on address 0x4816000000b0 at pc 0x5cb251414c17
READ of size 160 at 0x4816000000b0 tags: 05/02(65) (ptr/mem) in thread T2
Step 9 (test compiler-rt symbolizer) failure: test compiler-rt symbolizer (failure)
...
[379/384] Generating MSAN_INST_GTEST.gtest-all.cc.x86_64.o
[380/384] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.x86_64-with-call.o
[381/384] Generating Msan-x86_64-with-call-Test
[382/384] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.x86_64.o
[383/384] Generating Msan-x86_64-Test
[383/384] Running compiler_rt regression tests
llvm-lit: /home/b/sanitizer-x86_64-linux/build/llvm-project/llvm/utils/lit/lit/main.py:72: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds.
-- Testing: 10229 tests, 88 workers --
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
TIMEOUT: SanitizerCommon-hwasan-x86_64-Linux :: Posix/fork_threaded.c (10229 of 10229)
******************** TEST 'SanitizerCommon-hwasan-x86_64-Linux :: Posix/fork_threaded.c' FAILED ********************
Exit Code: -9
Timeout: Reached timeout of 900 seconds

Command Output (stderr):
--
RUN: at line 1: /home/b/sanitizer-x86_64-linux/build/build_default/./bin/clang  -gline-tables-only -fsanitize=hwaddress -fuse-ld=lld -fsanitize-hwaddress-experimental-aliasing  -m64 -funwind-tables  -I/home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test -ldl -O0 /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/sanitizer_common/TestCases/Posix/fork_threaded.c -o /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp && env HWASAN_OPTIONS=die_after_fork=0  /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp
+ /home/b/sanitizer-x86_64-linux/build/build_default/./bin/clang -gline-tables-only -fsanitize=hwaddress -fuse-ld=lld -fsanitize-hwaddress-experimental-aliasing -m64 -funwind-tables -I/home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test -ldl -O0 /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/sanitizer_common/TestCases/Posix/fork_threaded.c -o /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp
+ env HWASAN_OPTIONS=die_after_fork=0 /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp

=================================================================
==2129157==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 568 byte(s) in 3 object(s) allocated from:
    #0 0x5cb2514093c4 in malloc /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/lib/hwasan/hwasan_allocation_functions.cpp:151:3
    #1 0x5cb2515621fe in operator new(unsigned long) llvm-link
    #2 0x5cb2513ff9b4 in _start (/home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp+0xef9b4)

Direct leak of 204 byte(s) in 1 object(s) allocated from:
    #0 0x5cb251408f49 in calloc /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/lib/hwasan/hwasan_allocation_functions.cpp:127:3
    #1 0x5cb2514ea72f in llvm::safe_calloc(unsigned long, unsigned long) llvm-link
    #2 0x5cb2513ff9b4 in _start (/home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp+0xef9b4)

Indirect leak of 1792 byte(s) in 43 object(s) allocated from:
    #0 0x5cb251408995 in aligned_alloc /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/lib/hwasan/hwasan_allocation_functions.cpp:58:3
    #1 0x5cb2514f8019 in llvm::allocate_buffer(unsigned long, unsigned long) llvm-link

Indirect leak of 780 byte(s) in 1 object(s) allocated from:
    #0 0x5cb251408f49 in calloc /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/lib/hwasan/hwasan_allocation_functions.cpp:127:3
    #1 0x5cb2514ea72f in llvm::safe_calloc(unsigned long, unsigned long) llvm-link

Indirect leak of 160 byte(s) in 1 object(s) allocated from:
    #0 0x5cb2514093c4 in malloc /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/lib/hwasan/hwasan_allocation_functions.cpp:151:3
    #1 0x5cb2515621fe in operator new(unsigned long) llvm-link
    #2 0x5cb2513ff9b4 in _start (/home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp+0xef9b4)

SUMMARY: HWAddressSanitizer: 3504 byte(s) leaked in 49 allocation(s).
==2129157==ERROR: HWAddressSanitizer: tag-mismatch on address 0x4816000000b0 at pc 0x5cb251414c17
READ of size 160 at 0x4816000000b0 tags: 05/02(65) (ptr/mem) in thread T2

@ChuanqiXu9
Copy link
Member Author

I don't think this patch is related to Posix/fork_threaded.c in compiler-rt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:frontend Language frontend issues, e.g. anything involving "Sema" clang:modules C++20 modules and Clang Header Modules clang Clang issues not falling into any other category skip-precommit-approval PR for CI feedback, not intended for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[clang][Modules] Import then include fails to merge inline lambdas
3 participants