Skip to content

Implement runtime stacktraces #1344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
3a77c13
Add unwind_callback in lfortran intrinsics
Thirumalai-Shaktivel Dec 2, 2022
8df8fae
Implement get_stacktrace_addresses()
certik Dec 3, 2022
9a17532
Add print_stacktrace_addresses()
certik Dec 3, 2022
b1a930d
Implement call_print_stacktrace_addresses()
certik Dec 3, 2022
4f9c936
Implement get_local_addresses
Thirumalai-Shaktivel Dec 9, 2022
4c5f769
Add emit_loc for Assert in LLVM
Thirumalai-Shaktivel Dec 9, 2022
21868bd
Store binary_filename in the struct Stacktrace
Thirumalai-Shaktivel Dec 9, 2022
4d2fb08
Implement get_local_info_dwarfdump
Thirumalai-Shaktivel Dec 9, 2022
aa7a36b
Implement dat_convert.py
Thirumalai-Shaktivel Dec 9, 2022
d71d05c
Print Stacktrace
Thirumalai-Shaktivel Dec 9, 2022
400a591
Add colors and format the stacktrace
Thirumalai-Shaktivel Dec 10, 2022
ea7d029
Move link.h into the .c file
certik Dec 13, 2022
12e9f7f
Use HAVE_LFORTRAN_LINK in the .c file
certik Dec 13, 2022
60f9786
Fix a warning with %lld
certik Dec 13, 2022
d4fae4b
Put the increment on a separate line
certik Dec 13, 2022
2646e26
Move all the stacktrace related items to .c file,
Thirumalai-Shaktivel Dec 14, 2022
5375e0b
Print stacktrace in assert during the exit
Thirumalai-Shaktivel Dec 14, 2022
5b5afaf
Add print_stacktrace for the Stop and ErrorStop node
Thirumalai-Shaktivel Dec 14, 2022
66f28cc
Update gitignore
Thirumalai-Shaktivel Dec 14, 2022
d09f7e6
Use command line argument to get the filename in dat_convert
Thirumalai-Shaktivel Dec 14, 2022
ab405e7
Generate the stacktrace files using a function
Thirumalai-Shaktivel Dec 14, 2022
63064ab
Print stacktrace without color and styles
Thirumalai-Shaktivel Dec 14, 2022
03fe068
Add debug_emit_loc in Stop and ErrorStop
Thirumalai-Shaktivel Dec 14, 2022
d9622a6
Add run_with_dbg in run_tests
Thirumalai-Shaktivel Dec 14, 2022
044b898
Pass the `filename` to `get_base_name` function
Thirumalai-Shaktivel Jan 8, 2023
c655231
Use `strtol` to convert string to long int
Thirumalai-Shaktivel Jan 8, 2023
485cfd7
Use `dsymutil` to generate the required files for stacktrace in macOS
Thirumalai-Shaktivel Jan 8, 2023
66d017d
Implement `get_local_address` for macOS
Thirumalai-Shaktivel Jan 8, 2023
338eb0a
Define the arguments for the print_stacktrace function
Thirumalai-Shaktivel Jan 8, 2023
bb56119
Store the source_filename and use it to get the base_name
Thirumalai-Shaktivel Jan 8, 2023
a547e64
Get only the executable file's local_pc
Thirumalai-Shaktivel Jan 8, 2023
cb6f365
Fix the format specifier warnings
Thirumalai-Shaktivel Jan 8, 2023
268c430
Print the correct the stacktrace information,
Thirumalai-Shaktivel Jan 8, 2023
4dc7c2d
Use Libasr as a target_include_directories for lpython_runtime
Thirumalai-Shaktivel Jan 11, 2023
b8f648b
Increase the malloc size for the command
Thirumalai-Shaktivel Jan 11, 2023
58f133a
Enclose the runtime stacktrace handling within the HAVE_RUNTIME_STACK…
Thirumalai-Shaktivel Jan 12, 2023
59df7fd
Enable Runtime Stacktrace in the CI
Thirumalai-Shaktivel Jan 12, 2023
3708557
Generate the runtime_stacktrace related files during the compile-time
Thirumalai-Shaktivel Jan 12, 2023
b9d0738
Add a warning to use an option to print correct location info
Thirumalai-Shaktivel Jan 12, 2023
f00fe46
Add a test and update refs.
Thirumalai-Shaktivel Jan 12, 2023
70323bb
[MacOS] Ignore other file information except the source binary
Thirumalai-Shaktivel Jan 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ jobs:
echo "LFORTRAN_CMAKE_GENERATOR=Unix Makefiles" >> $GITHUB_ENV
echo "WIN=0" >> $GITHUB_ENV
echo "MACOS=0" >> $GITHUB_ENV
echo "ENABLE_RUNTIME_STACKTRACE=yes" >> $GITHUB_ENV

- name: Setup Platform (macOS)
if: contains(matrix.os, 'macos')
Expand All @@ -71,6 +72,7 @@ jobs:
echo "LFORTRAN_CMAKE_GENERATOR=Unix Makefiles" >> $GITHUB_ENV
echo "WIN=0" >> $GITHUB_ENV
echo "MACOS=1" >> $GITHUB_ENV
echo "ENABLE_RUNTIME_STACKTRACE=yes" >> $GITHUB_ENV

- name: Build (Linux / macOS)
shell: bash -l {0}
Expand All @@ -88,6 +90,7 @@ jobs:
set LFORTRAN_CMAKE_GENERATOR=Ninja
set WIN=1
set MACOS=0
set ENABLE_RUNTIME_STACKTRACE=no
call "C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Auxiliary/Build/vcvars64.bat"
xonsh ci\build.xsh

Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,9 @@ inst/bin/*
*.sln
*.dll
*.manifest
*_lines.txt
*_ldd.txt
*_lines.dat.txt

### https://raw.github.com/github/gitignore/218a941be92679ce67d0484547e3e142b2f5f6f0/Global/macOS.gitignore

Expand Down
9 changes: 9 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,15 @@ if (WITH_STACKTRACE)
endif()
set(HAVE_LFORTRAN_STACKTRACE yes)
endif()
if (WITH_RUNTIME_STACKTRACE)
set(WITH_UNWIND yes)
if (APPLE)
set(WITH_MACHO yes)
else()
set(WITH_LINKH yes)
endif()
set(HAVE_RUNTIME_STACKTRACE yes)
endif()
if (WITH_BFD)
find_package(BFD REQUIRED)
set(HAVE_LFORTRAN_BFD yes)
Expand Down
1 change: 1 addition & 0 deletions build1.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ cmake \
-DWITH_LLVM=yes \
-DLPYTHON_BUILD_ALL=yes \
-DWITH_STACKTRACE=yes \
-DWITH_RUNTIME_STACKTRACE=yes \
-DWITH_LSP=no \
-DWITH_LFORTRAN_BINARY_MODFILES=no \
-DCMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH_LPYTHON;$CONDA_PREFIX" \
Expand Down
2 changes: 1 addition & 1 deletion ci/build.xsh
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ cd test-bld
# compiled in Release mode and we get link failures if we mix and match build
# modes:
BUILD_TYPE = "Release"
cmake -G $LFORTRAN_CMAKE_GENERATOR -DCMAKE_VERBOSE_MAKEFILE=ON -DWITH_LLVM=yes -DWITH_LSP=yes -DWITH_XEUS=yes -DCMAKE_PREFIX_PATH=$CONDA_PREFIX -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX -DWITH_LFORTRAN_BINARY_MODFILES=no -DCMAKE_BUILD_TYPE=@(BUILD_TYPE) ..
cmake -G $LFORTRAN_CMAKE_GENERATOR -DCMAKE_VERBOSE_MAKEFILE=ON -DWITH_LLVM=yes -DWITH_LSP=yes -DWITH_XEUS=yes -DCMAKE_PREFIX_PATH=$CONDA_PREFIX -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX -DWITH_LFORTRAN_BINARY_MODFILES=no -DCMAKE_BUILD_TYPE=@(BUILD_TYPE) -DWITH_RUNTIME_STACKTRACE=$ENABLE_RUNTIME_STACKTRACE ..
cmake --build . --target install -j16
./src/lpython/tests/test_lpython
#./src/bin/lpython < ../src/bin/example_input.txt
Expand Down
7 changes: 7 additions & 0 deletions run_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ def is_included(backend):
c = is_included("c")
wat = is_included("wat")
run = is_included("run")
run_with_dbg = is_included("run_with_dbg")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a test here that we are running on Linux. Otherwise let's not run this test, since it doesn't work yet on other platforms.

pass_ = test.get("pass", None)
optimization_passes = ["flip_sign", "div_to_mul", "fma", "sign_from_value",
"inline_function_calls", "loop_unroll",
Expand Down Expand Up @@ -117,5 +118,11 @@ def is_included(backend):
run_test(filename, "runtime", "lpython {infile}",
filename, update_reference, extra_args)

if run_with_dbg:
run_test(
filename, "run_dbg",
"lpython {infile} -g --debug-with-line-column --no-color",
filename, update_reference, extra_args)

if __name__ == "__main__":
tester_main("LPython", single_test)
18 changes: 18 additions & 0 deletions src/bin/dat_convert.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/usr/bin/env python3

from struct import unpack
from sys import argv
from re import sub

lines = ""
with open(argv[1], "rb") as f:
lines = f.read()

list = []
for i in range(0, len(lines), 24):
list.append(sub('[(),]', '', str(unpack("3Q", lines[i:i+24]))))

with open(argv[1] + ".txt", "w") as f:
j = 0
for i in list:
f.write(i+'\n')
23 changes: 23 additions & 0 deletions src/bin/lpython.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1638,6 +1638,29 @@ int main(int argc, char *argv[])
err = link_executable({tmp_o}, outfile, runtime_library_dir,
backend, static_link, true, compiler_options);
if (err != 0) return err;

if (compiler_options.emit_debug_info) {
// TODO: Replace the following hardcoded part
std::string cmd = "";
#ifdef HAVE_LFORTRAN_MACHO
cmd += "dsymutil " + basename + ".out && llvm-dwarfdump --debug-line "
+ basename + ".out.dSYM > ";
#else
cmd += "llvm-dwarfdump --debug-line " + basename + ".out > ";
#endif
cmd += basename + "_ldd.txt && (cd src/bin; ./dwarf_convert.py ../../"
+ basename + "_ldd.txt ../../" + basename + "_lines.txt ../../"
+ basename + "_lines.dat && ./dat_convert.py ../../"
+ basename + "_lines.dat)";
int status = system(cmd.c_str());
if ( status != 0 ) {
std::cerr << "Error in creating the files used to generate "
"the debug information. This might be caused because either"
"`llvm-dwarfdump` or `Python` are not available. "
"Please activate the CONDA environment and compile again.\n";
return status;
}
}
#else
std::cerr << "Compiling Python files to object files requires the LLVM backend to be enabled. Recompile with `WITH_LLVM=yes`." << std::endl;
return 1;
Expand Down
26 changes: 25 additions & 1 deletion src/libasr/codegen/asr_to_llvm.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5165,8 +5165,16 @@ class ASRToLLVMVisitor : public ASR::BaseVisitor<ASRToLLVMVisitor>
}

void visit_Assert(const ASR::Assert_t &x) {
if (compiler_options.emit_debug_info) debug_emit_loc(x);
this->visit_expr_wrapper(x.m_test, true);
create_if_else(tmp, []() {}, [=]() {
if (compiler_options.emit_debug_info) {
llvm::Value *fmt_ptr = builder->CreateGlobalStringPtr(infile);
llvm::Value *fmt_ptr1 = llvm::ConstantInt::get(context, llvm::APInt(
1, compiler_options.use_colors));
call_print_stacktrace_addresses(context, *module, *builder,
{fmt_ptr, fmt_ptr1});
}
if (x.m_msg) {
char* s = ASR::down_cast<ASR::StringConstant_t>(x.m_msg)->m_s;
llvm::Value *fmt_ptr = builder->CreateGlobalStringPtr("AssertionError: %s\n");
Expand Down Expand Up @@ -5949,6 +5957,14 @@ class ASRToLLVMVisitor : public ASR::BaseVisitor<ASRToLLVMVisitor>
}

void visit_Stop(const ASR::Stop_t &x) {
if (compiler_options.emit_debug_info) {
debug_emit_loc(x);
llvm::Value *fmt_ptr = builder->CreateGlobalStringPtr(infile);
llvm::Value *fmt_ptr1 = llvm::ConstantInt::get(context, llvm::APInt(
1, compiler_options.use_colors));
call_print_stacktrace_addresses(context, *module, *builder,
{fmt_ptr, fmt_ptr1});
}
llvm::Value *fmt_ptr = builder->CreateGlobalStringPtr("STOP\n");
print_error(context, *module, *builder, {fmt_ptr});
llvm::Value *exit_code;
Expand All @@ -5963,7 +5979,15 @@ class ASRToLLVMVisitor : public ASR::BaseVisitor<ASRToLLVMVisitor>
exit(context, *module, *builder, exit_code);
}

void visit_ErrorStop(const ASR::ErrorStop_t & /* x */) {
void visit_ErrorStop(const ASR::ErrorStop_t &x) {
if (compiler_options.emit_debug_info) {
debug_emit_loc(x);
llvm::Value *fmt_ptr = builder->CreateGlobalStringPtr(infile);
llvm::Value *fmt_ptr1 = llvm::ConstantInt::get(context, llvm::APInt(
1, compiler_options.use_colors));
call_print_stacktrace_addresses(context, *module, *builder,
{fmt_ptr, fmt_ptr1});
}
llvm::Value *fmt_ptr = builder->CreateGlobalStringPtr("ERROR STOP\n");
print_error(context, *module, *builder, {fmt_ptr});
int exit_code_int = 1;
Expand Down
21 changes: 21 additions & 0 deletions src/libasr/codegen/llvm_utils.h
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,27 @@ namespace LFortran {
builder.CreateCall(fn_exit, {exit_code});
}

// Insert the following anywhere inside the LLVM backend to print
// addresses at runtime:
// call_print_stacktrace_addresses(context, *module, *builder, {filename, use_colors});
static inline void call_print_stacktrace_addresses(llvm::LLVMContext &context,
llvm::Module &module, llvm::IRBuilder<> &builder,
const std::vector<llvm::Value*> &args)
{
llvm::Function *fn = module.getFunction("print_stacktrace_addresses");
if (!fn) {
llvm::FunctionType *function_type = llvm::FunctionType::get(
llvm::Type::getVoidTy(context), {
llvm::Type::getInt8PtrTy(context),
llvm::Type::getInt1Ty(context)
}, true);
fn = llvm::Function::Create(function_type,
llvm::Function::ExternalLinkage, "print_stacktrace_addresses",
&module);
}
builder.CreateCall(fn, args);
}

namespace LLVM {

llvm::Value* CreateLoad(llvm::IRBuilder<> &builder, llvm::Value *x);
Expand Down
1 change: 1 addition & 0 deletions src/libasr/config.h.in
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@

/* Define if stacktrace is enabled */
#cmakedefine HAVE_LFORTRAN_STACKTRACE
#cmakedefine HAVE_RUNTIME_STACKTRACE
#cmakedefine HAVE_LFORTRAN_BFD
#cmakedefine HAVE_LFORTRAN_DWARFDUMP
#cmakedefine HAVE_LFORTRAN_LINK
Expand Down
Loading