[SingleSource/Vectorizer] Add unit tests for the vplan-native path. #20

iamlouk · 2023-09-08T08:22:04Z

This patch adds unit tests for LLVM's VPlan-native path, as requested in https://reviews.llvm.org/D157484.

The new Vectorizer/VPlanNativePath subdirectory is only enabled if the compiler is clang. For all source files in that directory, the flags "-mllvm -enable-vplan-native-path" are added.

Four different scenarios are tested for outer-loop vectorization:

Matrix multiplication, where the second of three loops is vectorized.
A test where the vectorized loop has an auxiliary induction variable.
A test for indirect and strided memory accesses.
A nesting of three loops where the outer-most one is vectorized.

This patch adds unit tests for LLVM's VPlan-native path, as requested in https://reviews.llvm.org/D157484. The new Vectorizer/VPlanNativePath subdirectory is only enabled if the compiler is clang. For all source files in that directory, the flags "-mllvm -enable-vplan-native-path" are added. Four different scenarios are tested for outer-loop vectorization: - Matrix multiplication, where the second of three loops is vectorized. - A test where the vectorized loop has an auxiliary induction variable. - A test for indirect and strided memory accesses. - A nesting of three loops where the outer-most one is vectorized.

iamlouk · 2023-09-08T08:28:59Z

I do not seam to be able to add reviewers, so sorry for tagging you here in a comment: @fhahn @alexey-bataev

Please also note that the matrix multiplication test will cause current LLVM to crash without https://reviews.llvm.org/D150700 which is not yet merged/accepted.

fhahn · 2023-09-22T11:34:51Z

Sorry for the delay with reviewing this!

Trying to build this with the latest version of main triggers an assert:

Assertion failed: ((From->getParent() == To->getParent()) && "Can't connect two block with different parents"), function connectBlocks, file VPlan.h, line 2840.

I'll get a reproducer.

iamlouk · 2023-09-22T11:42:38Z

No worries, thank you for having a look. I cannot confirm if its exactly that assert that breaks because I do not have my computer near by, but I know that the bug fixed in this patch is triggered by one of the tests: https://reviews.llvm.org/D150700 ("[LV] Stability fix for outerloop vectorization") by @nikolaypanchenko (@npanchen ? Sorry if tagging a random stranger, GitHub suggests this username).

fhahn · 2023-09-27T12:12:37Z

No worries, thank you for having a look. I cannot confirm if its exactly that assert that breaks because I do not have my computer near by, but I know that the bug fixed in this patch is triggered by one of the tests: https://reviews.llvm.org/D150700 ("[LV] Stability fix for outerloop vectorization") by @nikolaypanchenko (@npanchen ? Sorry if tagging a random stranger, GitHub suggests this username).

Indeed, looks like the patch should help, but would need a rebase.

fhahn · 2023-11-20T15:22:28Z

@iamlouk could you check if the new tests now build and run without issues after llvm/llvm-project#68118 landed?

iamlouk · 2023-11-21T09:32:01Z

Hello, yes, the tests compile and pass now!

I tested with llvm-project commit 88f0e4c75c1ac498f2223fc640c4ff6c572c5ed1, the llvm-test-suite was built using cmake -DCMAKE_C_COMPILER=<...>/bin/clang -DCMAKE_CXX_COMPILER=<...>/bin/clang++ -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_FLAGS="-mcpu=native" -DCMAKE_CXX_FLAGS="-mcpu=native" .. on a AArch64 host with SVE.

fhahn

LGTM, thanks!

An additional suggestion about keeping the variable names consistent inline.

fhahn · 2023-11-21T09:40:28Z

SingleSource/UnitTests/Vectorizer/VPlanNativePath/outer-loop-vect.cpp

+    DEFINE_SCALAR_AND_VECTOR_FN_FOR_NESTED_OLV(
+      (size_t N, size_t M, size_t L,
+       int32_t *__restrict__ A, const int32_t *B, const int32_t *C),
+      for (size_t j = 0; j < L; j++) {


Would be good to consistently use upper case for variable names, as per LLVM coding style.

Sorry for this, I fixed that!

fhahn

Thanks for the latest update, still LGTM!

iamlouk · 2023-12-04T21:21:24Z

Sorry to ask you for this, but could you merge this for me? I don't have the permissions. Thank you very much in advance.

SixWeining · 2024-01-23T12:57:31Z

Hi @iamlouk

I find that this test causes compliation crash on x86-64 with clang-18 O2:

clang++: /home/ray/llvm-clang/github-main/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:3149: llvm::BasicBlock *llvm::InnerLoopVectorizer::completeLoopSkeleton(): Assertion `DT->verify(DominatorTree::VerificationLevel::Fast)' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.	Program arguments: /home/ray/_install/bin/clang++ -DNDEBUG -O2 -w -Werror=date-time -mllvm -enable-vplan-native-path -fno-vectorize -MD -MT SingleSource/UnitTests/Vectorizer/VPlanNativePath/CMakeFiles/outer-loop-vect.dir/outer-loop-vect.cpp.o -MF SingleSource/UnitTests/Vectorizer/VPlanNativePath/CMakeFiles/outer-loop-vect.dir/outer-loop-vect.cpp.o.d -o SingleSource/UnitTests/Vectorizer/VPlanNativePath/CMakeFiles/outer-loop-vect.dir/outer-loop-vect.cpp.o -c /home/ray/test-suite/SingleSource/UnitTests/Vectorizer/VPlanNativePath/outer-loop-vect.cpp
1.	<eof> parser at end of file
2.	Optimizer
 #0 0x00007fbf6c23e3d7 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x483e3d7)
 #1 0x00007fbf6c23beae llvm::sys::RunSignalHandlers() (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x483beae)
 #2 0x00007fbf6c23d80d llvm::sys::CleanupOnSignal(unsigned long) (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x483d80d)
 #3 0x00007fbf6c175018 CrashRecoverySignalHandler(int) CrashRecoveryContext.cpp:0:0
 #4 0x00007fbf6745a510 (/lib/x86_64-linux-gnu/libc.so.6+0x3c510)
 #5 0x00007fbf674a80fc __pthread_kill_implementation ./nptl/pthread_kill.c:44:76
 #6 0x00007fbf6745a472 raise ./signal/../sysdeps/posix/raise.c:27:6
 #7 0x00007fbf674444b2 abort ./stdlib/abort.c:81:7
 #8 0x00007fbf674443d5 _nl_load_domain ./intl/loadmsgcat.c:1177:9
 #9 0x00007fbf674533a2 (/lib/x86_64-linux-gnu/libc.so.6+0x353a2)
#10 0x00007fbf6da9aa3e llvm::InnerLoopVectorizer::completeLoopSkeleton() (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x609aa3e)
#11 0x00007fbf6da9ab1c llvm::InnerLoopVectorizer::createVectorizedLoopSkeleton(llvm::DenseMap<llvm::SCEV const*, llvm::Value*, llvm::DenseMapInfo<llvm::SCEV const*, void>, llvm::detail::DenseMapPair<llvm::SCEV const*, llvm::Value*>> const&) (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x609ab1c)
#12 0x00007fbf6dabae4f llvm::LoopVectorizationPlanner::executePlan(llvm::ElementCount, unsigned int, llvm::VPlan&, llvm::InnerLoopVectorizer&, llvm::DominatorTree*, bool, llvm::DenseMap<llvm::SCEV const*, llvm::Value*, llvm::DenseMapInfo<llvm::SCEV const*, void>, llvm::detail::DenseMapPair<llvm::SCEV const*, llvm::Value*>> const*) (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x60bae4f)
#13 0x00007fbf6daccc83 llvm::LoopVectorizePass::processLoop(llvm::Loop*) (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x60ccc83)
#14 0x00007fbf6dad3bfd llvm::LoopVectorizePass::runImpl(llvm::Function&, llvm::ScalarEvolution&, llvm::LoopInfo&, llvm::TargetTransformInfo&, llvm::DominatorTree&, llvm::BlockFrequencyInfo*, llvm::TargetLibraryInfo*, llvm::DemandedBits&, llvm::AssumptionCache&, llvm::LoopAccessInfoManager&, llvm::OptimizationRemarkEmitter&, llvm::ProfileSummaryInfo*) (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x60d3bfd)
#15 0x00007fbf6dad45b4 llvm::LoopVectorizePass::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x60d45b4)
#16 0x00007fbf7039d65d llvm::detail::PassModel<llvm::Function, llvm::LoopVectorizePass, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) PassBuilder.cpp:0:0
#17 0x00007fbf6c441e18 llvm::PassManager<llvm::Function, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x4a41e18)
#18 0x00007fbf6ed8693d llvm::detail::PassModel<llvm::Function, llvm::PassManager<llvm::Function, llvm::AnalysisManager<llvm::Function>>, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) AMDGPUTargetMachine.cpp:0:0
#19 0x00007fbf6c447230 llvm::ModuleToFunctionPassAdaptor::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x4a47230)
#20 0x00007fbf6ed866fd llvm::detail::PassModel<llvm::Module, llvm::ModuleToFunctionPassAdaptor, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) AMDGPUTargetMachine.cpp:0:0
#21 0x00007fbf6c440966 llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x4a40966)
#22 0x00007fbf7386bf5e (anonymous namespace)::EmitAssemblyHelper::RunOptimizationPipeline(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>&, std::unique_ptr<llvm::ToolOutputFile, std::default_delete<llvm::ToolOutputFile>>&, clang::BackendConsumer*) BackendUtil.cpp:0:0
#23 0x00007fbf73862104 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>, clang::BackendConsumer*) (/home/ray/_install/bin/../lib/libclang-cpp.so.18git+0x2e62104)
#24 0x00007fbf73d0ddd5 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/home/ray/_install/bin/../lib/libclang-cpp.so.18git+0x330ddd5)
#25 0x00007fbf72428f63 clang::ParseAST(clang::Sema&, bool, bool) (/home/ray/_install/bin/../lib/libclang-cpp.so.18git+0x1a28f63)
#26 0x00007fbf7488bbc0 clang::FrontendAction::Execute() (/home/ray/_install/bin/../lib/libclang-cpp.so.18git+0x3e8bbc0)
#27 0x00007fbf747f89c2 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/home/ray/_install/bin/../lib/libclang-cpp.so.18git+0x3df89c2)
#28 0x00007fbf74917937 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/home/ray/_install/bin/../lib/libclang-cpp.so.18git+0x3f17937)
#29 0x0000000000218de7 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/home/ray/_install/bin/clang+++0x218de7)
#30 0x00000000002156ca ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&, llvm::ToolContext const&) driver.cpp:0:0
#31 0x00007fbf743a3a59 void llvm::function_ref<void ()>::callback_fn<clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const::$_1>(long) Job.cpp:0:0
#32 0x00007fbf6c174d58 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/home/ray/_install/bin/../lib/libLLVM-18git.so+0x4774d58)
#33 0x00007fbf743a3400 clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const (/home/ray/_install/bin/../lib/libclang-cpp.so.18git+0x39a3400)
#34 0x00007fbf7435b836 clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&, bool) const (/home/ray/_install/bin/../lib/libclang-cpp.so.18git+0x395b836)
#35 0x00007fbf7435bda7 clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&, bool) const (/home/ray/_install/bin/../lib/libclang-cpp.so.18git+0x395bda7)
#36 0x00007fbf74380e37 clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&) (/home/ray/_install/bin/../lib/libclang-cpp.so.18git+0x3980e37)
#37 0x0000000000214242 clang_main(int, char**, llvm::ToolContext const&) (/home/ray/_install/bin/clang+++0x214242)
#38 0x00000000002257d7 main (/home/ray/_install/bin/clang+++0x2257d7)
#39 0x00007fbf674456ca __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#40 0x00007fbf67445785 call_init ./csu/../csu/libc-start.c:128:20
#41 0x00007fbf67445785 __libc_start_main ./csu/../csu/libc-start.c:347:5
#42 0x000000000021172a _start (/home/ray/_install/bin/clang+++0x21172a)
clang++: error: clang frontend command failed with exit code 134 (use -v to see invocation)
clang version 18.0.0git (https://github.com/llvm/llvm-project ab32a3c166e9eae12260b4c6eca9bcf21f500e87)
Target: x86_64-unknown-linux-gnu
Thread model: posix

Did I miss something?

fhahn · 2024-01-23T13:44:36Z

@SixWeining interesting, could you share the IR causing the crash? (-mllvm -print-on-crash -mllvm -print-module-scope)

SixWeining · 2024-01-24T01:07:00Z

@SixWeining interesting, could you share the IR causing the crash? (-mllvm -print-on-crash -mllvm -print-module-scope)

Uploaded: crash.ll.txt Thanks.

SixWeining · 2024-01-25T03:22:12Z

Note that llvm is built with LLVM_ENABLE_EXPENSIVE_CHECKS=ON. The OFF build doesn't have such issue.

fhahn · 2024-01-25T12:51:24Z

@SixWeining not sure how expensive checks impact the codegen, but the issue should be fixed by a04f6152914ea21f3068aaba9d8fc21d2e703d3e

SixWeining · 2024-01-26T01:20:49Z

@SixWeining not sure how expensive checks impact the codegen, but the issue should be fixed by a04f6152914ea21f3068aaba9d8fc21d2e703d3e

Unfortunately, still crash with the assert in llvm/lib/Transforms/Vectorize/LoopVectorize.cpp and this assert is only enabled on a LLVM_ENABLE_EXPENSIVE_CHECKS=ON build.

 3141 #ifdef EXPENSIVE_CHECKS
 3142   assert(DT->verify(DominatorTree::VerificationLevel::Fast));
 3143 #endif
 3144 
 3145   return LoopVectorPreHeader;
 3146 }

iamlouk · 2024-01-26T05:43:16Z

Hello, I think this crash is just because of this condition in VPlan::execute(): // We do not attempt to preserve DT for outer loop vectorization currently.. I don't think this is an actual problem, though, because the dominator tree is not a preserved analysis in case the VPlan-native path is enabled (see here). IHMO, it would be fine just adding a if (!EnableVPlanNativePath) in front of the assert highlighted by @SixWeining .

@fhahn , the changes you made in a04f6152 are what could be needed to be able to remove the "-fno-vectorize" I added here, but have not confirmed that yet.

fhahn self-requested a review September 8, 2023 13:48

fhahn approved these changes Nov 21, 2023

View reviewed changes

Fix coding-style

1382bbc

fhahn approved these changes Nov 25, 2023

View reviewed changes

fhahn merged commit 6dadc3d into llvm:main Dec 5, 2023

[SingleSource/Vectorizer] Add unit tests for the vplan-native path. #20

[SingleSource/Vectorizer] Add unit tests for the vplan-native path. #20

Uh oh!

Conversation

iamlouk commented Sep 8, 2023

Uh oh!

iamlouk commented Sep 8, 2023

Uh oh!

fhahn commented Sep 22, 2023

Uh oh!

iamlouk commented Sep 22, 2023

Uh oh!

fhahn commented Sep 27, 2023

Uh oh!

fhahn commented Nov 20, 2023

Uh oh!

iamlouk commented Nov 21, 2023

Uh oh!

fhahn left a comment

Choose a reason for hiding this comment

Uh oh!

fhahn Nov 21, 2023

Choose a reason for hiding this comment

Uh oh!

iamlouk Nov 21, 2023

Choose a reason for hiding this comment

Uh oh!

fhahn left a comment

Choose a reason for hiding this comment

Uh oh!

iamlouk commented Dec 4, 2023

Uh oh!

SixWeining commented Jan 23, 2024

Uh oh!

fhahn commented Jan 23, 2024

Uh oh!

SixWeining commented Jan 24, 2024

Uh oh!

SixWeining commented Jan 25, 2024

Uh oh!

fhahn commented Jan 25, 2024

Uh oh!

SixWeining commented Jan 26, 2024

Uh oh!

iamlouk commented Jan 26, 2024

Uh oh!

Uh oh!