You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Run the vendor/bin/mftf run:test, vendor/bin/mftf run:group or vendor/bin/mftf run:failed command with failing tests
Expected result
The command should return a non-zero exit code as errors occured
Actual result
The command returns an exit code of zero
This is especially relevant in a CI environment where many job setups depend on the exit code of the run commands. In this case, we had a job showing as green even though the MFTF tests in the job failed.
This also blocks command chaining in the CLI, e.g. vendor/bin/mftf run:test FailingTest >> test.log && echo "Test success" will always print out "Test success".
The text was updated successfully, but these errors were encountered:
Preconditions
Steps to reproduce
Run the
vendor/bin/mftf run:test
,vendor/bin/mftf run:group
orvendor/bin/mftf run:failed
command with failing testsExpected result
The command should return a non-zero exit code as errors occured
Actual result
The command returns an exit code of zero
This is especially relevant in a CI environment where many job setups depend on the exit code of the run commands. In this case, we had a job showing as green even though the MFTF tests in the job failed.
This also blocks command chaining in the CLI, e.g.
vendor/bin/mftf run:test FailingTest >> test.log && echo "Test success"
will always print out "Test success".The text was updated successfully, but these errors were encountered: