Description of problem: Please take a look at https://beaker.engineering.redhat.com/jobs/944722 While the tests in the end were not executed due to a problem with harness itself beaker considered them as successfully completed. E.g. <task avg_time="120" duration="0:01:02" id="30228775" name="/CoreOS/nss/Sanity/use-of-invalid-ecc-key" result="New" role="None" status="Completed" version="1.0-2"><rpm name="nss-CoreOS-nss-Sanity-use-of-invalid-ecc-key" path="/mnt/tests/CoreOS/nss/Sanity/use-of-invalid-ecc-key"/><roles><role value="None"><system value="ibm-p8-01-lp5.rhts.eng.bos.redhat.com"/></role></roles><params><param name="CASERUNS" value="9757140"/></params></task>
*** This bug has been marked as a duplicate of bug 714937 ***
Bug 714937 is about the fact that a recipe will be Completed even if some tasks Aborted. However this bug is about a different situation. If you see Karel's linked job the last 10 tasks in the recipe never ran because every line in the task runner script died like this: ["Event", "lose_item", "3e75a0ed-52d1-466b-b23a-d52b9055e51e", {"id": "7e6077a9-9104-4d93-9309-bccfc108e99e"}, null, {"data": "fips.c(143): OpenSSL internal error, assertion failed: FATAL FIPS SELFTEST FAILURE\n"}] ["Event", "lose_item", "32ead422-6abd-4d8c-8852-41e814d03523", {"id": "7e6077a9-9104-4d93-9309-bccfc108e99e"}, null, {"data": "/mnt/testarea/tmp5Oin3K.sh: line 54: 21171 Aborted (core dumped) ... ["Event", "lose_item", "147d6f4b-7b48-4b73-b51a-f6a8961548f0", {"id": "7e6077a9-9104-4d93-9309-bccfc108e99e"}, null, {"data": "fips.c(143): OpenSSL internal error, assertion failed: FATAL FIPS SELFTEST FAILURE\n"}] ["Event", "lose_item", "d377d439-58cb-43c2-9697-0c6929ed9627", {"id": "7e6077a9-9104-4d93-9309-bccfc108e99e"}, null, {"data": "/mnt/testarea/tmp5Oin3K.sh: line 54: 21778 Aborted (core dumped) beahsh abort_task -m \"$TESTRPMNAME was not installed.\"\n"}] ["Event", "end", "82317aeb-b034-44d3-8f91-c4843f8c3b2a", {}, null, {"task_id": "7e6077a9-9104-4d93-9309-bccfc108e99e", "rc": 134}] but beah still considered the task to be Complete.
Closing this issue as the problem is within beah and there is a much better harness available, restraint. Please use restraint for your test cases. Re-open if this issue is a blocker.