[test] Disable makeAllowedFCmpRegion due to taking too long #111056

aeubanks · 2024-10-03T20:20:51Z

This was added in #110082. It takes 32s in an optimized build of LLVM, and doesn't finish after multiple minutes in an -O0 build. It's iterating all possible fp values 3 levels deep. Disable until the test is fixed to not take so much time.

This was added in llvm#110082. It takes 32s in an optimized build of LLVM, and doesn't finish after multiple minutes in an -O0 build. It's iterating all possible fp values 3 levels deep. Disable until the test is fixed to not take so much time.

llvmbot · 2024-10-03T20:21:53Z

@llvm/pr-subscribers-llvm-ir

Author: Arthur Eubanks (aeubanks)

Changes

This was added in #110082. It takes 32s in an optimized build of LLVM, and doesn't finish after multiple minutes in an -O0 build. It's iterating all possible fp values 3 levels deep. Disable until the test is fixed to not take so much time.

Full diff: https://github.com/llvm/llvm-project/pull/111056.diff

1 Files Affected:

(modified) llvm/unittests/IR/ConstantFPRangeTest.cpp (+3)

diff --git a/llvm/unittests/IR/ConstantFPRangeTest.cpp b/llvm/unittests/IR/ConstantFPRangeTest.cpp
index 17a08207fe1ba0..ae0ea8069fff5f 100644
--- a/llvm/unittests/IR/ConstantFPRangeTest.cpp
+++ b/llvm/unittests/IR/ConstantFPRangeTest.cpp
@@ -443,6 +443,9 @@ TEST_F(ConstantFPRangeTest, MismatchedSemantics) {
 #endif
 
 TEST_F(ConstantFPRangeTest, makeAllowedFCmpRegion) {
+  // TODO: reenable once test is fixed to run more quickly.
+  GTEST_SKIP();
+
   for (auto Pred : FCmpInst::predicates()) {
     EnumerateConstantFPRanges(
         [Pred](const ConstantFPRange &CR) {

vporpo

I was thinking of guarding this under NDEBUG, but 32s in Release sounds too much.

dtcxzyw · 2024-10-04T02:03:08Z

I have sped up the test run from 30.22s -> 10.59s by ignoring the payload in NaN values. Is it acceptable?

vporpo · 2024-10-04T02:08:45Z

I have sped up the test run from 30.22s -> 10.59s by ignoring the payload in NaN values. Is it acceptable?

What about the debug build? If it is 3 times faster, it would still be more than 2 and a half minutes, which is still too long.

dtcxzyw · 2024-10-04T02:22:20Z

I have sped up the test run from 30.22s -> 10.59s by ignoring the payload in NaN values. Is it acceptable?

What about the debug build? If it is 3 times faster, it would still be more than 2 and a half minutes, which is still too long.

I don't have enough space to test the debug build. But even if I disable this test, it still takes 10.58s to run the other existing unit tests.

vporpo · 2024-10-04T02:50:24Z

I don't have enough space to test the debug build.

I tried it and it takes 64 seconds in Debug. The next slowest one is ConstantFPRangeTest.FPClassify at 13s.

dtcxzyw · 2024-10-04T08:25:24Z

The fix has been landed in 856774d.

vporpo · 2024-10-04T15:22:28Z

IRTests used to take 57s for 782 tests and it now takes 2m2s for 783 tests. This is more than 100% increase just from one test, so it is still an issue for the Debug build.

dtcxzyw · 2024-10-04T16:18:49Z

IRTests used to take 57s for 782 tests and it now takes 2m2s for 783 tests. This is more than 100% increase just from one test, so it is still an issue for the Debug build.

I am sorry I cannot make it faster on debug build.
@aeubanks Can you make these exhaustive tests guarded by #ifndef NDEBUG?

arsenm · 2024-10-04T16:27:54Z

I am sorry I cannot make it faster on debug build. @aeubanks Can you make these exhaustive tests guarded by #ifndef NDEBUG?

NDEBUG won't catch debug build, and this should still run in rel + asserts. EXPENSIVE_CHECKS maybe?

Maybe there is something else to speed it up?

dtcxzyw · 2024-10-04T17:00:12Z

Maybe there is something else to speed it up?

With debug build:

[----------] 12 tests from ConstantFPRangeTest
[ RUN      ] ConstantFPRangeTest.Basics
[       OK ] ConstantFPRangeTest.Basics (0 ms)
[ RUN      ] ConstantFPRangeTest.Equality
[       OK ] ConstantFPRangeTest.Equality (0 ms)
[ RUN      ] ConstantFPRangeTest.SingleElement
[       OK ] ConstantFPRangeTest.SingleElement (0 ms)
[ RUN      ] ConstantFPRangeTest.ExhaustivelyEnumerate
[       OK ] ConstantFPRangeTest.ExhaustivelyEnumerate (70 ms)
[ RUN      ] ConstantFPRangeTest.Enumerate
[       OK ] ConstantFPRangeTest.Enumerate (1 ms)
[ RUN      ] ConstantFPRangeTest.IntersectWith
[       OK ] ConstantFPRangeTest.IntersectWith (0 ms)
[ RUN      ] ConstantFPRangeTest.UnionWith
[       OK ] ConstantFPRangeTest.UnionWith (0 ms)
[ RUN      ] ConstantFPRangeTest.FPClassify
[       OK ] ConstantFPRangeTest.FPClassify (2719 ms)
[ RUN      ] ConstantFPRangeTest.Print
[       OK ] ConstantFPRangeTest.Print (0 ms)
[ RUN      ] ConstantFPRangeTest.NonCanonicalEmptySet
[       OK ] ConstantFPRangeTest.NonCanonicalEmptySet (246 ms)
[ RUN      ] ConstantFPRangeTest.MismatchedSemantics
[       OK ] ConstantFPRangeTest.MismatchedSemantics (1221 ms)
[ RUN      ] ConstantFPRangeTest.makeAllowedFCmpRegion
[       OK ] ConstantFPRangeTest.makeAllowedFCmpRegion (52500 ms)
[----------] 12 tests from ConstantFPRangeTest (56758 ms total)
...
[==========] 783 tests from 109 test suites ran. (107440 ms total)
[  PASSED  ] 783 tests.

Please give me one more day to do some profiling.

The next slowest one is ConstantFPRangeTest.FPClassify at 13s.

Setting Exhaustive=true will reduce the test time of ConstantFPRangeTest.FPClassify from 2719ms to 61ms.

dtcxzyw · 2024-10-05T02:52:12Z

|--90.39%--(anonymous namespace)::ConstantFPRangeTest_makeAllowedFCmpRegion_Test::TestBody()::{lambda(llvm::ConstantFPRange const&)#1}::operator()(llvm::ConstantFPRange const&) const::{lambda(llvm::APFloat const&)#1}::operator()
                          |                                |          |          
                          |                                |          |--74.99%--(anonymous namespace)::AnyOfValueInConstantFPRange<(anonymous namespace)::ConstantFPRangeTest_makeAllowedFCmpRegion_Test::TestBody()::{lambda(llvm::ConstantFPRange const&)#1}::operator()(llvm::ConstantFPRange const&) const::{lambda(llvm::APFloat const&)#1}::operator()(llvm::APFloat const) const::{lambda(llvm::APFloat const)#1}>
                          |                                |          |          |          
                          |                                |          |          |--56.08%--bool (anonymous namespace)::AnyOfValueInConstantFPRange<(anonymous namespace)::ConstantFPRangeTest_makeAllowedFCmpRegion_Test::TestBody()::{lambda(llvm::ConstantFPRange const&)#1}::operator()(llvm::ConstantFPRange const&) const::{lambda(llvm::APFloat const&)#1}::operator()(llvm::APFloat const) const::{lambda(llvm::APFloat const)#1}>(llvm::ConstantFPRange const&, (anonymous namespace)::ConstantFPRangeTest_makeAllowedFCmpRegion_Test::TestBody()::{lambda(llvm::ConstantFPRange const&)#1}::operator()(llvm::ConstantFPRange const&) const::{lambda(llvm::APFloat const&)#1}::operator()(llvm::APFloat const) const::{lambda(llvm::APFloat const)#1}, bool)::{lambda({lambda(llvm::ConstantFPRange const&)#1}&)#1}::operator()
                          |                                |          |          |          |          
                          |                                |          |          |          |--46.40%--(anonymous namespace)::strictNext
                          |                                |          |          |          |          |          
                          |                                |          |          |          |          |--40.51%--llvm::APFloat::next

I cannot make it faster. I tried to cache the result of strictNext. But it took ~50% more time than before :(

EXPENSIVE_CHECKS maybe?

How about using __OPTIMIZE__/_DEBUG as we does for LLVM_IS_DEBUG_BUILD?

llvm-project/llvm/lib/Support/CommandLine.cpp

Lines 2501 to 2542 in e6549b8

    
           #if defined(__GNUC__) 
        
           // GCC and GCC-compatible compilers define __OPTIMIZE__ when optimizations are 
        
           // enabled. 
        
           # if defined(__OPTIMIZE__) 
        
           #  define LLVM_IS_DEBUG_BUILD 0 
        
           # else 
        
           #  define LLVM_IS_DEBUG_BUILD 1 
        
           # endif 
        
           #elif defined(_MSC_VER) 
        
           // MSVC doesn't have a predefined macro indicating if optimizations are enabled. 
        
           // Use _DEBUG instead. This macro actually corresponds to the choice between 
        
           // debug and release CRTs, but it is a reasonable proxy. 
        
           # if defined(_DEBUG) 
        
           #  define LLVM_IS_DEBUG_BUILD 1 
        
           # else 
        
           #  define LLVM_IS_DEBUG_BUILD 0 
        
           # endif 
        
           #else 
        
           // Otherwise, for an unknown compiler, assume this is an optimized build. 
        
           # define LLVM_IS_DEBUG_BUILD 0 
        
           #endif 
        
           namespace { 
        
           class VersionPrinter { 
        
           public: 
        
             void print(std::vector<VersionPrinterTy> ExtraPrinters = {}) { 
        
               raw_ostream &OS = outs(); 
        
           #ifdef PACKAGE_VENDOR 
        
               OS << PACKAGE_VENDOR << " "; 
        
           #else 
        
               OS << "LLVM (http://llvm.org/):\n  "; 
        
           #endif 
        
               OS << PACKAGE_NAME << " version " << PACKAGE_VERSION << "\n  "; 
        
           #if LLVM_IS_DEBUG_BUILD 
        
               OS << "DEBUG build"; 
        
           #else 
        
               OS << "Optimized build"; 
        
           #endif 
        
           #ifndef NDEBUG 
        
               OS << " with assertions"; 
        
           #endif 
        
               OS << ".\n";

…ECKS` (#111276) Addresses comment #111056 (comment).

aeubanks requested review from arsenm, vporpo and dtcxzyw October 3, 2024 20:20

llvmbot added the llvm:ir label Oct 3, 2024

vporpo approved these changes Oct 3, 2024

View reviewed changes

dtcxzyw closed this Oct 4, 2024

vporpo reopened this Oct 4, 2024

dtcxzyw mentioned this pull request Oct 6, 2024

[ConstantFPRange][UnitTests] Guard exhaustive checks by EXPENSIVE_CHECKS #111276

Merged

dtcxzyw added a commit that referenced this pull request Oct 8, 2024

[ConstantFPRange][UnitTests] Guard exhaustive checks by `EXPENSIVE_CH…

db4874c

…ECKS` (#111276) Addresses comment #111056 (comment).

aeubanks closed this Oct 9, 2024

aeubanks deleted the test branch October 9, 2024 16:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[test] Disable makeAllowedFCmpRegion due to taking too long #111056

[test] Disable makeAllowedFCmpRegion due to taking too long #111056

aeubanks commented Oct 3, 2024

llvmbot commented Oct 3, 2024

vporpo left a comment

dtcxzyw commented Oct 4, 2024

vporpo commented Oct 4, 2024

dtcxzyw commented Oct 4, 2024

vporpo commented Oct 4, 2024

dtcxzyw commented Oct 4, 2024

vporpo commented Oct 4, 2024

dtcxzyw commented Oct 4, 2024

arsenm commented Oct 4, 2024

dtcxzyw commented Oct 4, 2024

dtcxzyw commented Oct 5, 2024

[test] Disable makeAllowedFCmpRegion due to taking too long #111056

[test] Disable makeAllowedFCmpRegion due to taking too long #111056

Conversation

aeubanks commented Oct 3, 2024

llvmbot commented Oct 3, 2024

vporpo left a comment

Choose a reason for hiding this comment

dtcxzyw commented Oct 4, 2024

vporpo commented Oct 4, 2024

dtcxzyw commented Oct 4, 2024

vporpo commented Oct 4, 2024

dtcxzyw commented Oct 4, 2024

vporpo commented Oct 4, 2024

dtcxzyw commented Oct 4, 2024

arsenm commented Oct 4, 2024

dtcxzyw commented Oct 4, 2024

dtcxzyw commented Oct 5, 2024