PR for llvm/llvm-project#57626 #152
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This speeds up processing for inputs with lots of missing metrics, by avoiding adding them to names. Reviewed By: paquette Differential Revision: https://reviews.llvm.org/D64065 llvm-svn: 366999
This fixes the following error when using --all.
Traceback (most recent call last):
File "test-suite/utils/compare.py", line 374, in <module>
print_result(data, limit_output, shorten_names, config.show_diff, sortkey)
File "test-suite/utils/compare.py", line 227, in print_result
dataout = add_geomean_row(d, dataout)
File "test-suite/utils/compare.py", line 139, in add_geomean_row
relative = values1 / values0
File "pyvenv/lib/python2.7/site-packages/pandas/core/ops.py", line 1583, in wrapper
result = safe_na_op(lvalues, rvalues)
File "pyvenv/lib/python2.7/site-packages/pandas/core/ops.py", line 1533, in safe_na_op
lambda x: op(x, rvalues))
File "pandas/_libs/algos.pyx", line 690, in pandas._libs.algos.arrmap
File "pyvenv/lib/python2.7/site-packages/pandas/core/ops.py", line 1533, in <lambda>
lambda x: op(x, rvalues))
TypeError: ufunc 'true_divide' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
Reviewed By: paquette
Differential Revision: https://reviews.llvm.org/D64066
llvm-svn: 367002
Summary: Memory functions (memcmp, memcpy, ...) are typically recognized by the compiler and expanded to specific asm patterns when the size is known at compile time. This will help catch any regressions in expansions. Right now we're only testing memcmp (see context in D60318). Subscribers: mgorny, llvm-commits, gchatelet, spatel Tags: #llvm Differential Revision: https://reviews.llvm.org/D64082 llvm-svn: 369707
We're using constexpr in the code, so this breaks in older systems. MicroBenchmarks/MemFunctions/main.cpp:34:10: error: unknown type name 'constexpr' static constexpr size_t kMaxBufSizeBytes = 4096; See https://reviews.llvm.org/D64082 llvm-svn: 369888
…ullptr, it's UB
Caught by D67122
/repositories/llvm-test-suite/MultiSource/Benchmarks/MiBench/consumer-typeset/z31.c:352:8: runtime error: applying non-zero offset 96 to null pointer
#0 0x2e682c in GetMemory (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-typeset+0x2e682c)
#1 0x258c4b in InitFiles (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-typeset+0x258c4b)
#2 0x24d34e in main (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-typeset+0x24d34e)
#3 0x7fe3fb2ac09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
#4 0x22e029 in _start (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-typeset+0x22e029)
After that if() the code expects to have enough memory,
so the branch should be unconditionally taken if next_free == nullptr,
llvm-svn: 371219
By adding negated unsigned offset, while with wrapping semantics
you indeed get a smaller pointer, this `op[]` operation is in-bounds
by the standards, and is lowered into `getelementpointer inbounds`,
and it is not wrapping. So the fact that we get the correct answer
does not matter, we got it illegally.
Fix it to actually perform the subtraction,
via using negative signed offset
/repositories/llvm-test-suite/MultiSource/Benchmarks/MallocBench/gs/zarray.c:75:2: runtime error: addition of unsigned offset to 0x000000b0faa0 overflowed to 0x000000b0fa70
#0 0x24ab6b in zastore (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MallocBench/gs/gs+0x24ab6b)
#1 0x245593 in interp (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MallocBench/gs/gs+0x245593)
#2 0x244b27 in interpret (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MallocBench/gs/gs+0x244b27)
#3 0x241a80 in run_file (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MallocBench/gs/gs+0x241a80)
#4 0x241a07 in init2 (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MallocBench/gs/gs+0x241a07)
#5 0x2416f1 in argproc (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MallocBench/gs/gs+0x2416f1)
#6 0x241d1c in gs_main (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MallocBench/gs/gs+0x241d1c)
#7 0x2412cd in main (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MallocBench/gs/gs+0x2412cd)
#8 0x7f76cde3409a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
#9 0x222029 in _start (/builddirs/build-test-suite-new/MultiSource/Benchmarks/MallocBench/gs/gs+0x222029)
Seems to be a preexisting issue, but found while evaluating D67122.
llvm-svn: 371220
Mirror LLVM's cmake llvm_codesign function into the test-suite and apply to all executables. llvm-svn: 372241
…B in C Caught by https://reviews.llvm.org/D67122 Address of the zero-th element of an array is identical to the address of the array base pointer (&a[0] == a), so let's just not apply zero offset. llvm-test-suite/MultiSource/Applications/d/parse.c:1588:20: runtime error: applying zero offset to null pointer #0 0x251be9 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x251be9) #1 0x2515b4 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2515b4) #2 0x2515b4 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2515b4) #3 0x2515b4 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2515b4) #4 0x2515b4 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2515b4) #5 0x2515b4 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2515b4) #6 0x2515b4 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2515b4) #7 0x2515b4 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2515b4) #8 0x2515b4 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2515b4) #9 0x2515b4 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2515b4) #10 0x2515b4 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2515b4) #11 0x2515b4 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2515b4) #12 0x2515b4 in commit_tree (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2515b4) #13 0x253e87 in commit_stack (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x253e87) #14 0x253e44 in commit_stack (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x253e44) #15 0x250ef1 in exhaustive_parse (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x250ef1) #16 0x24facd in dparse (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x24facd) #17 0x2441b0 in parse_grammar (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x2441b0) #18 0x23a270 in main (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x23a270) #19 0x7fac77faebba in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26bba) #20 0x21aeb9 in _start (/builddirs/build-test-suite-old/MultiSource/Applications/d/make_dparser+0x21aeb9) SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /repositories/llvm-test-suite/MultiSource/Applications/d/parse.c:1588:20 in llvm-svn: 373486
Summary: This patch adds support for testing clang/LLVM against the GCC Torture suite. This patch adds the CMake configuration and licence information for these tests. A follow-up patch will add the testcases themselves (which are too large to review, and included without modifications). They will be committed together. Reviewers: hfinkel, kristof.beyls, asb Reviewed By: kristof.beyls Subscribers: khcheang, mehdi_amini, jvesely, krytarowski, fedor.sergeev, zzheng, steven_wu, dexonsmith, arphaman, jfb, mstorsjo, lewis-revill, simoncook, s.egerton, riccibruno, asb, mgorny, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66887 llvm-svn: 374155
llvm-svn: 374156
Summary: The LLVM test suite has its own way of detecting the system architecture. This adds support in that file for detecting RISC-V. This will eventually cause the ARCH variable to be populated. We use ARCH="riscv64" to identify 64-bit RISC-V, and ARCH="riscv32" to identify 32-bit RISC-V, so that attempting to detect "riscv" in the ARCH variable will match any version of RISC-V. Reviewers: asb, luismarques Reviewed By: luismarques Subscribers: mgorny, simoncook, kito-cheng, shiva0217, rogfer01, rkruppe, PkmX, s.egerton, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68698 llvm-svn: 374176
llvm-svn: 374295
…command Summary: As of SPEC CPU2017 1.0.5, 526.blender_r/data does not contain an 'all' subdirectory, which breaks the build. Reviewers: Meinersbur, MatzeB Subscribers: mgorny, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68967 llvm-svn: 375480
Review: Ulrich Weigand
sollve_vv is the OpenMP Validation & Verification suite to test conformance to with the OpenMP specification. It is included as an external project, i.e. some work is required before it is enabled, for instance setting TEST_SUITE_SOLLVEVV_ROOT to the downloaded source repository. OpenMP is detected using CMake's find_package(OpenMP) but the offloading flags must be specified manually in CMake using TEST_SUITE_SOLLVEVV_OFFLOADING_CFLAGS and TEST_SUITE_SOLLVEVV_OFFLOADING_LDFLAGS. Differential Revision: https://reviews.llvm.org/D70065
include(External) is necessary with TEST_SUITE_SUBDIRS=External/sollve_vv. Before this patch, it was included by a different CMakeLists.txt processed before sollve_vv's.
Since the width of a bit field cannot exceed the width of the underlying type, we should change fraction's type to long long for ia32.
Fix issue on PowerPC that unix-smail fails to compile, which should be a name conflict about `optopt` variable.
Some of these tests use features which are not supported on Darwin. (E.g. aliases) As a result of this, there are some unhappy Darwin test suite bots. So, let's just blacklist those tests when we're targeting Darwin. Example of unhappy bot: http://green.lab.llvm.org/green/job/test-suite-verify-machineinstrs-x86_64-O0-g/
Follow conventions in https://reviews.llvm.org/D40075.
Summary: as suggested by `config/Example-aocc-linux-x86.cfg` (v1.1) or `config/Example-clang-llvm-linux-x86.cfg` (v1.0) Otherwise, it may fail to compile. Reviewers: Meinersbur, kristof.beyls, MatzeB Reviewed by: Meinersbur Differential Revision: https://reviews.llvm.org/D71684
Summary: There are some minor changes on the perf-stat output that needs to be accommodated so we parse the time data correctly. Reviewers: jmolloy, kongyi Reviewed by: kongyi Differential Revision: https://reviews.llvm.org/D72198
In the test-suite, we have two files: SingleSource/UnitTests/Vector/Altivec/ld.reference_output SingleSource/UnitTests/Vector/Altivec/ld.c That need to have their name changed to: SingleSource/UnitTests/Vector/Altivec/vec_ld.reference_output SingleSource/UnitTests/Vector/Altivec/vec_ld.c Because they cause other test cases in the directory to not produce executable once the ld test is built. The reason for the failure is that the ./ld is used as the linker when building other tests.
Sync fix from https://reviews.llvm.org/rL181270 to CMakefiles. In order for the test-suite outputs to match the reference outputs we need to specify -ffp-contract=off. Not only does extra FMA precision cause differences between the outputs and the reference outputs, but also between our output and that of gcc (because the two compilers sometimes make slightly different contraction choices). Fortunately, providing -ffp-contract=off to both compilers reveals agreement. Older versions of gcc did not support -ffp-contract, and if using such a version, you'll need to use the reference outputs (which you should be able to do now).
The NUM_BLOCK_TYPES macro is defined to 10, but the arrays in ctx_tables.h only have 8 elements in the relevant dimension, so this was causing out of bounds array accesses, and segfaults if we were (un)lucky. I don't know enough about H264 decoding to know if this is the right fix, but it fixes the segfault currently affecting the Arm LNT bot, and doesn't change the expected output. Differential revision: https://reviews.llvm.org/D74770
This was failing to build because of a libcxx change: https://reviews.llvm.org/D71214
This prevents all the variables being declared as globals. I noticed this due to a typo while working on D121908.
In addition, this retains the pandas multi-index to improve the
visualization. We also transform the program column in-place now since
it appears that DataFrame.to_string() ignores the formatter argument for
all columns that have dtype=object.
Before: utils/compare.py a.json vs b.json --lhs-name=baseline --rhs-name=new -m compile_time -m link_time --nodiff
```
Program baseline new baseline new
-C++/Shootout-C++-lists 0.00 0.00 0.09 0.08
-C++/Shootout-C++-objinst 0.00 0.00 0.07 0.08
...
baseline new baseline new
count 315.000000 315.000000 315.000000 315.000000
mean 3.543040 3.531691 0.076500 0.078680
...
```
As can be seen here, the metric is missing from the column headers, so it's not
clear which one is which if you use multiple metrics.
With this change we now see the metrics, and they are sorted by largest diff:
```
Program compile_time link_time
baseline new baseline new
SingleSource/Benchmarks/Stanford/Perm 0.07 0.08 0.08 0.11
SingleSour...Benchmarks/Stanford/Bubblesort 0.07 0.08 0.10 0.10
...
compile_time link_time
l/r baseline new baseline new
count 315.000000 315.000000 315.000000 315.000000
mean 3.543040 3.531691 0.076500 0.078680
...
```
More importantly (my original motivation for this change), the script currently
fails to run when passing multiple metrics without --nodiff. For example,
`utils/compare.py a.json vs b.json -m compile_time -m link_time` result in:
`pandas.errors.InvalidIndexError: Reindexing only valid with uniquely valued Index objects`
I believe this used to work and regressed in 0ee69b8
(https://reviews.llvm.org/D57828) as the exception is raised in add_geomean_row()
After:
```
Program compile_time link_time
baseline new diff baseline new diff
SingleSource/Benchmarks/Stanford/Perm 0.07 0.08 26.2% 0.08 0.11 34.4%
SingleSour...Benchmarks/Stanford/Bubblesort 0.07 0.08 16.9% 0.10 0.10 4.0%
...
Geomean difference -0.4% 2.9%
compile_time link_time
l/r baseline new diff baseline new diff
count 315.000000 315.000000 283.000000 315.000000 315.000000 315.000000
mean 3.543040 3.531691 -0.002523 0.076500 0.078680 0.041652
...
```
Listing multiple metrics with a single file was also broken:
`utils/compare.py a.json -m compile_time -m link_time` results an exception
while executing `d['$sortkey'] = d[sortkey]`.
`ValueError: Expected a 1D array, got an array with shape (315, 2)`.
This problem has most likely existed since the initial commit that added this
script, but with this change we can now handle mutiple metrics using the newer
pandas `d.sort_values(by=(metrics[0], sortkey), ascending=False)` instead of
adding a new column and sorting on that. We now get a table sorted by the
first metric:
```
Program compile_time link_time
results results
ultiSource...Benchmarks/7zip/7zip-benchmark 109.56 0.22
ultiSource/Benchmarks/Bullet/bullet 73.55 0.17
...
compile_time link_time
run results results
count 315.000000 315.000000
mean 3.543040 0.076500
...
```
Reviewed By: MatzeB
Differential Revision: https://reviews.llvm.org/D121908
This is useful for metrics such as number of times a given pass performed a transformation, etc. Reviewed By: MatzeB Differential Revision: https://reviews.llvm.org/D124102
This adds CFLAGS as well as CXXFLAGS for disabling implicit function declarations in both CMake and Makefiles. Hopefully addresses the issues in: https://lab.llvm.org/buildbot/#/builders/105/builds/24314
This hopefully addresses the issue in: https://lab.llvm.org/buildbot/#/builders/57/builds/17109
This should address the issues found in: https://lab.llvm.org/buildbot/#/builders/214/builds/839
This hopefully addresses what was found by: https://lab.llvm.org/buildbot/#/builders/214/builds/840 It also guesses what the next diagnostic that builder will find is and speculatively fixes it.
This addresses the next failure found by: https://lab.llvm.org/buildbot/#/builders/214/builds/842
This should address the issue found by: https://lab.llvm.org/buildbot/#/builders/214/builds/844
This should address most (all?) of the issues found in: https://lab.llvm.org/buildbot/#/builders/105/builds/24922 https://lab.llvm.org/buildbot/#/builders/100/builds/15282
This should hopefully address the issues found by: https://lab.llvm.org/buildbot/#/builders/105/builds/24927 https://lab.llvm.org/buildbot/#/builders/174/builds/8906
The test is for the functionality implemented in https://reviews.llvm.org/D110235 Differential Revision: https://reviews.llvm.org/D124609
For FreeBSD, we also have to pass -DSPEC_CPU_MACOSX to avoid the use of gcvt() in 400.perlbench as this function is not available on FreeBSD. This define is also required to successfully build 462.libquantum and 483.xalancbmk since SPEC_CPU_BSD is not handled in those benchmarks. Reviewed By: Meinersbur, fhahn Differential Revision: https://reviews.llvm.org/D124393
Fixes an issue revealed by https://reviews.llvm.org/D121343, causing several of the test-suite-verify-machineinstrs* bots on GreenDragon to fail with: ld: library not found for -lrt rdar://91014785 Differential revision: https://reviews.llvm.org/D125275
The LLVM test suite has its own way of detecting the system architecture. This adds support in that file for detecting C-SKY. Differential Revision: https://reviews.llvm.org/D100924
Upstream clang supports RDC-mode compilation through the new driver. This patch adds an option that allows the user to configure the compilation job to use the new driver for testing. This will allow us to optionally test the new driver in the buildbots to track changes as we develop. Reviewed By: tra Differential Revision: https://reviews.llvm.org/D126231
Use gettimeofday to get the number of cycles since power-on in CSKY backend. Differential Revision: https://reviews.llvm.org/D100926
Disable XSBench and RSBench running on CSKY. Because the biggest alignment of CSKY is 4 bytes so that the expected Est. Memory Usage is different with reference output. Differential Revision: https://reviews.llvm.org/D126453
By default, Flang outputs 'Fortran STOP' as part of the STOP statement, and sometimes also warnings about signaling exceptions, which are platform-dependent. This output will cause test failures, but it can be suppressed by setting the environment variable NO_STOP_MESSAGE=1. This patch makes sure that llvm-lit forwards the NO_STOP_MESSAGE variable to the tests if it is set in the environment. Differential Revision: https://reviews.llvm.org/D126360
At the moment, flang doesn't support `-Werror=date-time`, so we shouldn't add it if the Fortran compiler is set to flang. This shouldn't affect any other compiler (and especially not C/C++ ones). Differential Revision: https://reviews.llvm.org/D126951
The char* here isn't 4 byte aligned so the cast to int32_t* can create an unaligned access. Replace with a memcpy. Differential Revision: https://reviews.llvm.org/D128491
Currently, when compiling with the CMake flag TEST_SUITE_PROFILE_GENERATE set to ON, compare-fp-3.c fails to compile as clang tries to insert instrumentation on the functions that are supposed to get optimized out. This commit fixes this by adding references to these functions when profile generation is turned on so that the test suite builds without linking errors. Differential Revision: https://reviews.llvm.org/D128225
Some tests in the FCVS suite use list-directed output, which may according to the standard introduce arbitrary amounts of whitespace. This patch adds the "-i" flag to the fpcmp invocation for those tests, so that it ignores whitespace. The other tests in the suite will continue to check whitespace strictly. Differential Revision: https://reviews.llvm.org/D128260
Unless using the `-a` or `-r` options, do not parse floating point numbers in order to compare its numeric difference. Despite its name, `fpcmp` used used to compare every program output with its reference output even for programs that do not handle floating point and caused `0.` and `0` to be always considered equal. I the old behaviour is needed (as for the TSVC benchmark), floating point comparison with zero numeric tolerance can be enabled by explicitly setting `-a 0` or `-r 0`. Motivated by D128262. Reviewed By: MatzeB Differential Revision: https://reviews.llvm.org/D129017
This updates the test suite to ignore -Wint-conversion warnings which have been upgraded to an error by https://reviews.llvm.org/D129881. Differential Revision: https://reviews.llvm.org/D129891
This should address the issue found by: https://lab.llvm.org/buildbot/#/builders/214/builds/2499
This patch adds dedicated test coverage for multi order recurrences. D119661 will add support for vectorizing the second and third order tests. Reviewed By: Meinersbur Differential Revision: https://reviews.llvm.org/D129339
As reported in Issue #57626, `frame_layout.cpp` doesn't compile on SPARC (both Solaris/sparcv9 and Linux/sparc64): FAILED: MultiSource/UnitTests/C++11/frame_layout/CMakeFiles/frame_layout.dir/frame_layout.cpp.o [...] fatal error: error in backend: Function "_Z10test_frameILj4096ELb1ELj2048ELb0ELb0EJEEvDpT4_jPKcP13trace_elementd": over-aligned dynamic alloca not supported. void test_frame<4096u, true, 2048u, false, false>(, unsigned int, char const*, trace_element*, double) Therefore, this patch disables the test on SPARC. Tested on `sparc64-unknown-linux-gnu` (only 3 testsuite failures remaining) and `sparcv9-sun-solaris2.11` (11 failures once a couple of Solaris-specific bugs are fixed). Differential Revision: https://reviews.llvm.org/D133493
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
resolves llvm/llvm-project#57626