Skip to content

[clang] Merge gtest binaries into AllClangUnitTests #134196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Apr 29, 2025

Conversation

rnk
Copy link
Collaborator

@rnk rnk commented Apr 3, 2025

This reduces the size of the clang/unittests build directory by 64% and my overall build dir size by 5%. Static linking is the real driving factor here, but even if the default build configuration used shared libraries, I don't see why we should be building so many unit test binaries.

To make the project more approachable for new contributors, I'm attempting to make the build a bit less resource-intensive. Build directory size is a common complaint, and this is low-hanging fruit.

This reduces the size of the clang/unittests build directory by 64% and
my overall build dir size by 5%. Static linking is the real driving
factor here, but even if the default build configuration used shared
libraries, I don't see why we should be building so many unit test
binaries.

To make the project more approachable for new contributors, I'm
attempting to make the build a bit less resource-intensive. Build
directory size is a common complaint, and this is low-hanging fruit.
@rnk rnk requested a review from Copilot April 3, 2025 04:13
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR renames the gtest binary for unit tests to reduce build directory size and overall resource usage.

  • Renames a test case from ModuleCacheTest to DriverModuleCacheTest
  • Aligns unit test naming with the goal of merging binaries
Files not reviewed (3)
  • clang/unittests/CMakeLists.txt: Language not supported
  • clang/unittests/Interpreter/CMakeLists.txt: Language not supported
  • clang/unittests/Interpreter/ExceptionTests/CMakeLists.txt: Language not supported
Comments suppressed due to low confidence (1)

clang/unittests/Driver/ModuleCacheTest.cpp:20

  • [nitpick] The test case name 'DriverModuleCacheTest' is inconsistent with the file name 'ModuleCacheTest.cpp'. Consider renaming either the file or the test case for clarity.
TEST(DriverModuleCacheTest, GetTargetAndMode) {

@llvmbot llvmbot added clang Clang issues not falling into any other category clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' labels Apr 3, 2025
@llvmbot
Copy link
Member

llvmbot commented Apr 3, 2025

@llvm/pr-subscribers-clang-codegen

@llvm/pr-subscribers-clang-driver

Author: Reid Kleckner (rnk)

Changes

This reduces the size of the clang/unittests build directory by 64% and my overall build dir size by 5%. Static linking is the real driving factor here, but even if the default build configuration used shared libraries, I don't see why we should be building so many unit test binaries.

To make the project more approachable for new contributors, I'm attempting to make the build a bit less resource-intensive. Build directory size is a common complaint, and this is low-hanging fruit.


Full diff: https://github.com/llvm/llvm-project/pull/134196.diff

4 Files Affected:

  • (modified) clang/unittests/CMakeLists.txt (+51-2)
  • (modified) clang/unittests/Driver/ModuleCacheTest.cpp (+1-1)
  • (modified) clang/unittests/Interpreter/CMakeLists.txt (+1-1)
  • (modified) clang/unittests/Interpreter/ExceptionTests/CMakeLists.txt (+1-1)
diff --git a/clang/unittests/CMakeLists.txt b/clang/unittests/CMakeLists.txt
index f3823ba309420..8d4476761e03e 100644
--- a/clang/unittests/CMakeLists.txt
+++ b/clang/unittests/CMakeLists.txt
@@ -15,11 +15,11 @@ if(CLANG_BUILT_STANDALONE)
   endif()
 endif()
 
-# add_clang_unittest(test_name file1.cpp file2.cpp)
+# add_distinct_clang_unittest(test_name file1.cpp file2.cpp)
 #
 # Will compile the list of files together and link against the clang
 # Produces a binary named 'basename(test_name)'.
-function(add_clang_unittest test_name)
+function(add_distinct_clang_unittest test_name)
   cmake_parse_arguments(ARG
     ""
     ""
@@ -47,6 +47,33 @@ function(add_clang_unittest test_name)
   target_link_libraries(${test_name} PRIVATE ${ARG_LINK_LIBS})
 endfunction()
 
+define_property(GLOBAL PROPERTY CLANG_UNITTEST_SRCS)
+define_property(GLOBAL PROPERTY CLANG_UNITTEST_LLVM_COMPONENTS)
+define_property(GLOBAL PROPERTY CLANG_UNITTEST_CLANG_LIBS)
+define_property(GLOBAL PROPERTY CLANG_UNITTEST_LINK_LIBS)
+
+# add_clang_unittest(test_name file1.cpp file2.cpp)
+#
+# Adds unittests to the combined AllClangUnitTests binary. The unittest binary
+# is defined after adding all unittest subdirectories.
+function(add_clang_unittest test_name)
+  cmake_parse_arguments(ARG
+    ""
+    ""
+    "CLANG_LIBS;LINK_LIBS;LLVM_COMPONENTS"
+    ${ARGN})
+
+  file(RELATIVE_PATH src_prefix "${CMAKE_CURRENT_FUNCTION_LIST_DIR}" "${CMAKE_CURRENT_SOURCE_DIR}")
+  set(srcs_prefixed)
+  foreach(src ${ARG_UNPARSED_ARGUMENTS})
+    set(srcs_prefixed ${srcs_prefixed} "${src_prefix}/${src}")
+  endforeach()
+  set_property(GLOBAL APPEND PROPERTY CLANG_UNITTEST_SRCS ${srcs_prefixed})
+  set_property(GLOBAL APPEND PROPERTY CLANG_UNITTEST_CLANG_LIBS ${ARG_CLANG_LIBS})
+  set_property(GLOBAL APPEND PROPERTY CLANG_UNITTEST_LINK_LIBS ${ARG_LINK_LIBS})
+  set_property(GLOBAL APPEND PROPERTY CLANG_UNITTEST_LLVM_COMPONENTS ${ARG_LLVM_COMPONENTS})
+endfunction()
+
 add_subdirectory(Basic)
 add_subdirectory(Lex)
 add_subdirectory(Parse)
@@ -77,3 +104,25 @@ add_subdirectory(Index)
 add_subdirectory(InstallAPI)
 add_subdirectory(Serialization)
 add_subdirectory(Support)
+
+
+# If we're doing a single merged clang unit test binary, add that target after
+# all the previous subdirectories have been processed.
+get_property(SRCS GLOBAL PROPERTY CLANG_UNITTEST_SRCS)
+get_property(CLANG_LIBS GLOBAL PROPERTY CLANG_UNITTEST_CLANG_LIBS)
+get_property(LINK_LIBS GLOBAL PROPERTY CLANG_UNITTEST_LINK_LIBS)
+get_property(LLVM_COMPONENTS GLOBAL PROPERTY CLANG_UNITTEST_LLVM_COMPONENTS)
+add_distinct_clang_unittest(AllClangUnitTests
+  ${SRCS}
+  CLANG_LIBS
+  ${CLANG_LIBS}
+  LINK_LIBS
+  ${LINK_LIBS}
+  LLVM_COMPONENTS
+  ${LLVM_COMPONENTS}
+)
+
+# The Tooling library has some internal headers. Make those work. If we like
+# the merged clang unit test binary, we can udpate the include paths and make
+# this the default.
+include_directories(Tooling)
diff --git a/clang/unittests/Driver/ModuleCacheTest.cpp b/clang/unittests/Driver/ModuleCacheTest.cpp
index 48744415647e6..7aa5047c59bd3 100644
--- a/clang/unittests/Driver/ModuleCacheTest.cpp
+++ b/clang/unittests/Driver/ModuleCacheTest.cpp
@@ -17,7 +17,7 @@ using namespace clang::driver;
 
 namespace {
 
-TEST(ModuleCacheTest, GetTargetAndMode) {
+TEST(DriverModuleCacheTest, GetTargetAndMode) {
   SmallString<128> Buf;
   Driver::getDefaultModuleCachePath(Buf);
   StringRef Path = Buf;
diff --git a/clang/unittests/Interpreter/CMakeLists.txt b/clang/unittests/Interpreter/CMakeLists.txt
index 9df1a4b03da47..1dda9024075a1 100644
--- a/clang/unittests/Interpreter/CMakeLists.txt
+++ b/clang/unittests/Interpreter/CMakeLists.txt
@@ -1,4 +1,4 @@
-add_clang_unittest(ClangReplInterpreterTests
+add_distinct_clang_unittest(ClangReplInterpreterTests
   IncrementalCompilerBuilderTest.cpp
   IncrementalProcessingTest.cpp
   InterpreterTest.cpp
diff --git a/clang/unittests/Interpreter/ExceptionTests/CMakeLists.txt b/clang/unittests/Interpreter/ExceptionTests/CMakeLists.txt
index eb366a860661c..dfd94d8e6442c 100644
--- a/clang/unittests/Interpreter/ExceptionTests/CMakeLists.txt
+++ b/clang/unittests/Interpreter/ExceptionTests/CMakeLists.txt
@@ -3,7 +3,7 @@
 set(LLVM_REQUIRES_EH ON)
 set(LLVM_REQUIRES_RTTI ON)
 
-add_clang_unittest(ClangReplInterpreterExceptionTests
+add_distinct_clang_unittest(ClangReplInterpreterExceptionTests
   InterpreterExceptionTest.cpp
   EXPORT_SYMBOLS
 

@llvmbot
Copy link
Member

llvmbot commented Apr 3, 2025

@llvm/pr-subscribers-clang

Author: Reid Kleckner (rnk)

Changes

This reduces the size of the clang/unittests build directory by 64% and my overall build dir size by 5%. Static linking is the real driving factor here, but even if the default build configuration used shared libraries, I don't see why we should be building so many unit test binaries.

To make the project more approachable for new contributors, I'm attempting to make the build a bit less resource-intensive. Build directory size is a common complaint, and this is low-hanging fruit.


Full diff: https://github.com/llvm/llvm-project/pull/134196.diff

4 Files Affected:

  • (modified) clang/unittests/CMakeLists.txt (+51-2)
  • (modified) clang/unittests/Driver/ModuleCacheTest.cpp (+1-1)
  • (modified) clang/unittests/Interpreter/CMakeLists.txt (+1-1)
  • (modified) clang/unittests/Interpreter/ExceptionTests/CMakeLists.txt (+1-1)
diff --git a/clang/unittests/CMakeLists.txt b/clang/unittests/CMakeLists.txt
index f3823ba309420..8d4476761e03e 100644
--- a/clang/unittests/CMakeLists.txt
+++ b/clang/unittests/CMakeLists.txt
@@ -15,11 +15,11 @@ if(CLANG_BUILT_STANDALONE)
   endif()
 endif()
 
-# add_clang_unittest(test_name file1.cpp file2.cpp)
+# add_distinct_clang_unittest(test_name file1.cpp file2.cpp)
 #
 # Will compile the list of files together and link against the clang
 # Produces a binary named 'basename(test_name)'.
-function(add_clang_unittest test_name)
+function(add_distinct_clang_unittest test_name)
   cmake_parse_arguments(ARG
     ""
     ""
@@ -47,6 +47,33 @@ function(add_clang_unittest test_name)
   target_link_libraries(${test_name} PRIVATE ${ARG_LINK_LIBS})
 endfunction()
 
+define_property(GLOBAL PROPERTY CLANG_UNITTEST_SRCS)
+define_property(GLOBAL PROPERTY CLANG_UNITTEST_LLVM_COMPONENTS)
+define_property(GLOBAL PROPERTY CLANG_UNITTEST_CLANG_LIBS)
+define_property(GLOBAL PROPERTY CLANG_UNITTEST_LINK_LIBS)
+
+# add_clang_unittest(test_name file1.cpp file2.cpp)
+#
+# Adds unittests to the combined AllClangUnitTests binary. The unittest binary
+# is defined after adding all unittest subdirectories.
+function(add_clang_unittest test_name)
+  cmake_parse_arguments(ARG
+    ""
+    ""
+    "CLANG_LIBS;LINK_LIBS;LLVM_COMPONENTS"
+    ${ARGN})
+
+  file(RELATIVE_PATH src_prefix "${CMAKE_CURRENT_FUNCTION_LIST_DIR}" "${CMAKE_CURRENT_SOURCE_DIR}")
+  set(srcs_prefixed)
+  foreach(src ${ARG_UNPARSED_ARGUMENTS})
+    set(srcs_prefixed ${srcs_prefixed} "${src_prefix}/${src}")
+  endforeach()
+  set_property(GLOBAL APPEND PROPERTY CLANG_UNITTEST_SRCS ${srcs_prefixed})
+  set_property(GLOBAL APPEND PROPERTY CLANG_UNITTEST_CLANG_LIBS ${ARG_CLANG_LIBS})
+  set_property(GLOBAL APPEND PROPERTY CLANG_UNITTEST_LINK_LIBS ${ARG_LINK_LIBS})
+  set_property(GLOBAL APPEND PROPERTY CLANG_UNITTEST_LLVM_COMPONENTS ${ARG_LLVM_COMPONENTS})
+endfunction()
+
 add_subdirectory(Basic)
 add_subdirectory(Lex)
 add_subdirectory(Parse)
@@ -77,3 +104,25 @@ add_subdirectory(Index)
 add_subdirectory(InstallAPI)
 add_subdirectory(Serialization)
 add_subdirectory(Support)
+
+
+# If we're doing a single merged clang unit test binary, add that target after
+# all the previous subdirectories have been processed.
+get_property(SRCS GLOBAL PROPERTY CLANG_UNITTEST_SRCS)
+get_property(CLANG_LIBS GLOBAL PROPERTY CLANG_UNITTEST_CLANG_LIBS)
+get_property(LINK_LIBS GLOBAL PROPERTY CLANG_UNITTEST_LINK_LIBS)
+get_property(LLVM_COMPONENTS GLOBAL PROPERTY CLANG_UNITTEST_LLVM_COMPONENTS)
+add_distinct_clang_unittest(AllClangUnitTests
+  ${SRCS}
+  CLANG_LIBS
+  ${CLANG_LIBS}
+  LINK_LIBS
+  ${LINK_LIBS}
+  LLVM_COMPONENTS
+  ${LLVM_COMPONENTS}
+)
+
+# The Tooling library has some internal headers. Make those work. If we like
+# the merged clang unit test binary, we can udpate the include paths and make
+# this the default.
+include_directories(Tooling)
diff --git a/clang/unittests/Driver/ModuleCacheTest.cpp b/clang/unittests/Driver/ModuleCacheTest.cpp
index 48744415647e6..7aa5047c59bd3 100644
--- a/clang/unittests/Driver/ModuleCacheTest.cpp
+++ b/clang/unittests/Driver/ModuleCacheTest.cpp
@@ -17,7 +17,7 @@ using namespace clang::driver;
 
 namespace {
 
-TEST(ModuleCacheTest, GetTargetAndMode) {
+TEST(DriverModuleCacheTest, GetTargetAndMode) {
   SmallString<128> Buf;
   Driver::getDefaultModuleCachePath(Buf);
   StringRef Path = Buf;
diff --git a/clang/unittests/Interpreter/CMakeLists.txt b/clang/unittests/Interpreter/CMakeLists.txt
index 9df1a4b03da47..1dda9024075a1 100644
--- a/clang/unittests/Interpreter/CMakeLists.txt
+++ b/clang/unittests/Interpreter/CMakeLists.txt
@@ -1,4 +1,4 @@
-add_clang_unittest(ClangReplInterpreterTests
+add_distinct_clang_unittest(ClangReplInterpreterTests
   IncrementalCompilerBuilderTest.cpp
   IncrementalProcessingTest.cpp
   InterpreterTest.cpp
diff --git a/clang/unittests/Interpreter/ExceptionTests/CMakeLists.txt b/clang/unittests/Interpreter/ExceptionTests/CMakeLists.txt
index eb366a860661c..dfd94d8e6442c 100644
--- a/clang/unittests/Interpreter/ExceptionTests/CMakeLists.txt
+++ b/clang/unittests/Interpreter/ExceptionTests/CMakeLists.txt
@@ -3,7 +3,7 @@
 set(LLVM_REQUIRES_EH ON)
 set(LLVM_REQUIRES_RTTI ON)
 
-add_clang_unittest(ClangReplInterpreterExceptionTests
+add_distinct_clang_unittest(ClangReplInterpreterExceptionTests
   InterpreterExceptionTest.cpp
   EXPORT_SYMBOLS
 

@rnk rnk marked this pull request as draft April 3, 2025 04:53
@rnk rnk marked this pull request as ready for review April 10, 2025 14:42
@llvmbot llvmbot added the clang:codegen IR generation bugs: mangling, exceptions, etc. label Apr 10, 2025
Copy link

github-actions bot commented Apr 10, 2025

✅ With the latest revision this PR passed the C/C++ code formatter.

@rnk
Copy link
Collaborator Author

rnk commented Apr 11, 2025

This is ready now, so please take a look. Apologies for all the push notifications. It took a lot of retries to get this to pass the premerge tests due to cmake environmental differences, which I guess shows the value of shared premerge testing.

@rnk
Copy link
Collaborator Author

rnk commented Apr 25, 2025

Ping, WDYT?

@rnk
Copy link
Collaborator Author

rnk commented Apr 29, 2025

I disabled the failing test in 743c32e.

@nico
Copy link
Contributor

nico commented Apr 30, 2025

BasicTests and FormatTests had fairly few deps (they didn't need Sema etc) and compiled much fewer files than they presumably now do. Does it make sense to keep those as distinct binaries?

nico added a commit that referenced this pull request Apr 30, 2025
The CMake build merged most clang unittests into a single binary.
The gn build doesn't (yet?) do this, but as part of that change,
FrontendTests somehow picked up a dependency on AllTargetsDescs,
see #134196 (comment)
@rnk
Copy link
Collaborator Author

rnk commented Apr 30, 2025

BasicTests and FormatTests had fairly few deps (they didn't need Sema etc) and compiled much fewer files than they presumably now do. Does it make sense to keep those as distinct binaries?

FormatTests has many clang dependencies, but after rebuilding, it looks like it's still quite small, so I sent a PR with my file size data:
#138021

You can see that the Interpreter tests are a much bigger target for optimization.

@owenca
Copy link
Contributor

owenca commented May 1, 2025

This patch caused an assertion failure for b8bb1cc on several build bots and my mac (M2, 15.4.1):

[ RUN      ] CodeGenTest.TestNonAlterTest
Assertion failed: (MRI && "Unable to create reg info"), function initAsmInfo, file CodeGenTargetMachineImpl.cpp, line 48.

It also made us lose the FormatTests build target:

$ ninja FormatTests
ninja: error: unknown target 'FormatTests'

Update:
Back to normal after afd738c.

@rnk
Copy link
Collaborator Author

rnk commented May 1, 2025

Are the CodeGenTest.TestNonAlterTest failures still an issue? I don't see that failure on any bots. I haven't gotten any reports.

@owenca
Copy link
Contributor

owenca commented May 2, 2025

@rnk see #137577 (comment) for some of the bots that got the assertion failure.

Update: Back to normal after afd738c.

I meant the assertion failure also disappeared after that commit.

@AaronBallman
Copy link
Collaborator

AaronBallman commented May 2, 2025

@rnk see #137577 (comment) for some of the bots that got the assertion failure.

Update: Back to normal after afd738c.

I meant the assertion failure also disappeared after that commit.

FWIW, I'm seeing the test failure on Windows x64 with a debug build of Clang's unit tests as of a fresh fetch of main this morning.

The failing test is:

TEST(CodeGenTest, CodeGenFromIRMemBuffer) {
  auto Invocation = std::make_shared<CompilerInvocation>();
  std::unique_ptr<MemoryBuffer> MemBuffer =
      MemoryBuffer::getMemBuffer("", "test.ll");
  Invocation->getFrontendOpts().Inputs.push_back(
      FrontendInputFile(*MemBuffer, Language::LLVM_IR));
  Invocation->getFrontendOpts().ProgramAction = frontend::EmitLLVMOnly;
  Invocation->getTargetOpts().Triple = "i386-unknown-linux-gnu";
  CompilerInstance Compiler(std::move(Invocation));
  Compiler.createDiagnostics(*llvm::vfs::getRealFileSystem());
  EXPECT_TRUE(Compiler.hasDiagnostics());

  EmitLLVMOnlyAction Action;
  bool Success = Compiler.ExecuteAction(Action);
  EXPECT_TRUE(Success);
}

and the failure is this assertion:

void CodeGenTargetMachineImpl::initAsmInfo() {
  MRI.reset(TheTarget.createMCRegInfo(getTargetTriple().str()));
  assert(MRI && "Unable to create reg info");
  ...

@rnk
Copy link
Collaborator Author

rnk commented May 2, 2025

FWIW, I'm seeing the test failure on Windows x64 with a debug build of Clang's unit tests as of a fresh fetch of main this morning.

Sorry about that! I leaned on the Windows premerge tests to find Windows issues (they did, that was useful), but I'm pretty sure it uses release, not debug, builds.

I won't be able to repro any windows build failures until Monday, and it's likely I'll be buried in other stuff.

If it's a singular test case, I would disable the test (rename it to add the DISABLED_ prefix to the test name as in 743c32e) to unblock work. Reverting and relanding will create new stale gtest executables in everyone's build directories.

If you want to go ahead and debug it, the next steps I would try are adding all the various LLVMInitializeAll* API calls to the gtest test case. I think what's going on here is that the TargetInfo classes are registered, but the MCTargetDesc or register info classes are not registered. This happens because some other gtest case running earlier registers the targets, and now the CodeGenTest links in the LLVM Target backends, when previously they were not present in the old gtest binary.

@dyung
Copy link
Collaborator

dyung commented May 2, 2025

I've been seeing intermittent failures of AllClangUnitTests/TimeProfilerTest/ConstantEvaluationCxx20 and AllClangUnitTests/TimeProfilerTest/ConstantEvaluationC99 on my MacOS bot, but I haven't had a chance to try and figure out if it might be related to your change or not.

ConstantEvaluationCxx20:

First instance of either failing: https://lab.llvm.org/buildbot/#/builders/190/builds/19216

@aeubanks
Copy link
Contributor

aeubanks commented May 3, 2025

I've also seen ConstantEvaluationC99 intermittently fail on our mac bots recently

@AaronBallman
Copy link
Collaborator

Yup, seeing this again on #132776

@rnk
Copy link
Collaborator Author

rnk commented May 5, 2025

I hacked together a workaround here: #138613

I'm mostly just guessing remotely, since I don't have a reliable repro, that the final PerformPending... event is rounding into the previous event group that just ended. If the JSON encodes this nesting relationship already, it might be nice to expose that to deflake the test more.

IanWood1 pushed a commit to IanWood1/llvm-project that referenced this pull request May 6, 2025
This reduces the size of the clang/unittests build directory by 64% and
my overall build dir size by 5%. Static linking is the real driving
factor here, but even if the default build configuration used shared
libraries, I don't see why we should be building so many unit test
binaries.

To make the project more approachable for new contributors, I'm
attempting to make the build a bit less resource-intensive. Build
directory size is a common complaint, and this is low-hanging fruit.

I've noticed that incremental builds leave behind the old, stale gtest binaries, and lit will keep running them. This mostly doesn't matter unless they use shared libraries, which will eventually stop working after successive builds. You can clean up the old test binaries with this command in the build directory:
  $ find tools/clang/unittests/ -iname '*Tests' -type f | xargs rm

... or you can simply clean the build directory in a more holistic way.

---------

Co-authored-by: Petr Hosek <[email protected]>
IanWood1 pushed a commit to IanWood1/llvm-project that referenced this pull request May 6, 2025
The CMake build merged most clang unittests into a single binary.
The gn build doesn't (yet?) do this, but as part of that change,
FrontendTests somehow picked up a dependency on AllTargetsDescs,
see llvm#134196 (comment)
IanWood1 pushed a commit to IanWood1/llvm-project that referenced this pull request May 6, 2025
This reduces the size of the clang/unittests build directory by 64% and
my overall build dir size by 5%. Static linking is the real driving
factor here, but even if the default build configuration used shared
libraries, I don't see why we should be building so many unit test
binaries.

To make the project more approachable for new contributors, I'm
attempting to make the build a bit less resource-intensive. Build
directory size is a common complaint, and this is low-hanging fruit.

I've noticed that incremental builds leave behind the old, stale gtest binaries, and lit will keep running them. This mostly doesn't matter unless they use shared libraries, which will eventually stop working after successive builds. You can clean up the old test binaries with this command in the build directory:
  $ find tools/clang/unittests/ -iname '*Tests' -type f | xargs rm

... or you can simply clean the build directory in a more holistic way.

---------

Co-authored-by: Petr Hosek <[email protected]>
IanWood1 pushed a commit to IanWood1/llvm-project that referenced this pull request May 6, 2025
The CMake build merged most clang unittests into a single binary.
The gn build doesn't (yet?) do this, but as part of that change,
FrontendTests somehow picked up a dependency on AllTargetsDescs,
see llvm#134196 (comment)
IanWood1 pushed a commit to IanWood1/llvm-project that referenced this pull request May 6, 2025
This reduces the size of the clang/unittests build directory by 64% and
my overall build dir size by 5%. Static linking is the real driving
factor here, but even if the default build configuration used shared
libraries, I don't see why we should be building so many unit test
binaries.

To make the project more approachable for new contributors, I'm
attempting to make the build a bit less resource-intensive. Build
directory size is a common complaint, and this is low-hanging fruit.

I've noticed that incremental builds leave behind the old, stale gtest binaries, and lit will keep running them. This mostly doesn't matter unless they use shared libraries, which will eventually stop working after successive builds. You can clean up the old test binaries with this command in the build directory:
  $ find tools/clang/unittests/ -iname '*Tests' -type f | xargs rm

... or you can simply clean the build directory in a more holistic way.

---------

Co-authored-by: Petr Hosek <[email protected]>
IanWood1 pushed a commit to IanWood1/llvm-project that referenced this pull request May 6, 2025
The CMake build merged most clang unittests into a single binary.
The gn build doesn't (yet?) do this, but as part of that change,
FrontendTests somehow picked up a dependency on AllTargetsDescs,
see llvm#134196 (comment)
llvm-sync bot pushed a commit to arm/arm-toolchain that referenced this pull request May 6, 2025
The CMake build merged most clang unittests into a single binary.
The gn build doesn't (yet?) do this, but as part of that change,
FrontendTests somehow picked up a dependency on AllTargetsDescs,
see llvm/llvm-project#134196 (comment)
GeorgeARM pushed a commit to GeorgeARM/llvm-project that referenced this pull request May 7, 2025
This reduces the size of the clang/unittests build directory by 64% and
my overall build dir size by 5%. Static linking is the real driving
factor here, but even if the default build configuration used shared
libraries, I don't see why we should be building so many unit test
binaries.

To make the project more approachable for new contributors, I'm
attempting to make the build a bit less resource-intensive. Build
directory size is a common complaint, and this is low-hanging fruit.

I've noticed that incremental builds leave behind the old, stale gtest binaries, and lit will keep running them. This mostly doesn't matter unless they use shared libraries, which will eventually stop working after successive builds. You can clean up the old test binaries with this command in the build directory:
  $ find tools/clang/unittests/ -iname '*Tests' -type f | xargs rm

... or you can simply clean the build directory in a more holistic way.

---------

Co-authored-by: Petr Hosek <[email protected]>
GeorgeARM pushed a commit to GeorgeARM/llvm-project that referenced this pull request May 7, 2025
The CMake build merged most clang unittests into a single binary.
The gn build doesn't (yet?) do this, but as part of that change,
FrontendTests somehow picked up a dependency on AllTargetsDescs,
see llvm#134196 (comment)
Ankur-0429 pushed a commit to Ankur-0429/llvm-project that referenced this pull request May 9, 2025
This reduces the size of the clang/unittests build directory by 64% and
my overall build dir size by 5%. Static linking is the real driving
factor here, but even if the default build configuration used shared
libraries, I don't see why we should be building so many unit test
binaries.

To make the project more approachable for new contributors, I'm
attempting to make the build a bit less resource-intensive. Build
directory size is a common complaint, and this is low-hanging fruit.

I've noticed that incremental builds leave behind the old, stale gtest binaries, and lit will keep running them. This mostly doesn't matter unless they use shared libraries, which will eventually stop working after successive builds. You can clean up the old test binaries with this command in the build directory:
  $ find tools/clang/unittests/ -iname '*Tests' -type f | xargs rm

... or you can simply clean the build directory in a more holistic way.

---------

Co-authored-by: Petr Hosek <[email protected]>
Ankur-0429 pushed a commit to Ankur-0429/llvm-project that referenced this pull request May 9, 2025
The CMake build merged most clang unittests into a single binary.
The gn build doesn't (yet?) do this, but as part of that change,
FrontendTests somehow picked up a dependency on AllTargetsDescs,
see llvm#134196 (comment)
@tahonermann
Copy link
Contributor

I ended up spending a lot of time barking down the wrong rabbit hole due to these changes. @rnk, please see #140799.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:codegen IR generation bugs: mangling, exceptions, etc. clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' clang Clang issues not falling into any other category
Projects
None yet
Development

Successfully merging this pull request may close these issues.