Print reason of crash to .reject for tap tests

[Korablev77]

Before this commit dac6b2b if sql-tap test crashed, the reason of crash (i.e. stack-trace or fired assertion) would be logged into *.reject file. It was great feature and I miss it so much. Moreover, it seems to be quite useful especially when executing test-run with force (--f) flag.

[Totktonada]

Another possible way to handle it: print last lines of a tarantool log (more then 15) right after last output when --force is enabled.

[Totktonada]

I tried it like so:

diff --git a/test/sql-tap/alias.test.lua b/test/sql-tap/alias.test.lua
index 57e3335e9..86d338106 100755
--- a/test/sql-tap/alias.test.lua
+++ b/test/sql-tap/alias.test.lua
@@ -1,4 +1,12 @@
 #!/usr/bin/env tarantool
+
+local ffi = require('ffi')
+ffi.cdef([[
+    void getpid(void);
+    int kill(pid_t pid, int sig);
+]])
+ffi.C.kill(ffi.C.getpid(), 11)
+
 test = require("sqltester")
 test:plan(9)
 
diff --git a/test/sql-tap/engine.cfg b/test/sql-tap/engine.cfg
index 413dd3ff4..d06096d33 100644
--- a/test/sql-tap/engine.cfg
+++ b/test/sql-tap/engine.cfg
@@ -11,6 +11,9 @@
     "sort.test.lua": {
         "memtx": {"engine": "memtx"}
     },
+    "alias.test.lua": {
+        "memtx": {"engine": "memtx"}
+    },
     "*": {
         "memtx": {"engine": "memtx"},
         "vinyl": {"engine": "vinyl"}

Run:

TEST_RUN_TESTS=sql-tap/alias make test-force

Got:

[001] sql-tap/alias.test.lua                          memtx           [ fail ]
[001] Test failed! Output from reject file alias.reject:
[001] 
[001] Last 15 lines of Tarantool Log file [Instance "app_server"][/home/alex/p/tarantool-meta/review/tarantool/test/var/001_sql-tap/alias.test.lua:memtx.tarantool.log]:
[001] #0  0x557e2516afa2 in print_backtrace+9
[001] #1  0x557e2504cf94 in _ZL12sig_fatal_cbiP9siginfo_tPv+1e7
[001] #2  0x7f74e88777e0 in funlockfile+80
[001] #3  0x7f74e7ec2897 in kill+7
[001] #4  0x557e2519175b in lj_vm_ffi_call+84
[001] #5  0x557e2522244f in lj_ccall_func+3e8
[001] #6  0x557e25248241 in lj_cf_ffi_meta___call+aa
[001] #7  0x557e2518f7ab in lj_BC_FUNCC+34
[001] #8  0x557e251b22f5 in lua_pcall+18b
[001] #9  0x557e25151f04 in luaT_call+29
[001] #10 0x557e2514a3bd in lua_main+b9
[001] #11 0x557e2514aa53 in run_script_f+60d
[001] #12 0x557e2504c97a in _ZL16fiber_cxx_invokePFiP13__va_list_tagES0_+1e
[001] #13 0x557e25167dbe in fiber_loop+82
[001] #14 0x557e25330e9a in coro_init+4c

@Korablev77 So the issue is resolved now?

[Totktonada]

I think result/tmp_result/reject files are about stdout of a test and should not contain stderr/log.

We can introduce separate status file for failing tests that will contain a test output, stderr/logs and maybe other related info.

Another way is to show diff between result tmp_result (instead of result and reject) and make a reject file this status file.

Anyway, we need to reconsider stdout/stderr/logs handling at the whole and pay attention to TAP13 tests (#68).

[Totktonada]

I see no problems here now.

stderr is written to a separate file, which is shown on the screen, when a fail occurs.

If anyone experience problems with stderr, please, describe them as a user scenario and reopen the issue. I also suggest to wait for PR #394 first.