Bug 812320 - when phase is not finished, although FAIL was reported, test ends as PASS
when phase is not finished, although FAIL was reported, test ends as PASS
Status: CLOSED WONTFIX
Product: Fedora
Classification: Fedora
Component: beakerlib (Show other bugs)
16
Unspecified Unspecified
unspecified Severity high
: ---
: ---
Assigned To: Petr Muller
Fedora Extras Quality Assurance
:
Depends On:
Blocks: 545868
  Show dependency treegraph
 
Reported: 2012-04-13 07:35 EDT by Jan Hutař
Modified: 2016-09-19 22:08 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-02-13 21:15:48 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jan Hutař 2012-04-13 07:35:38 EDT
Description of problem:
When, because of test creator mistake, phase is not finished by rlPhaseEnd, although FAIL was reported earlier in the phase, test finishes as PASS.


Version-Release number of selected component (if applicable):
Current


How reproducible:
always


Steps to Reproduce:
1. Consider this (of-course, this is incorrect, but test should not PASS):
   for i in 1 2 3; do
     rlPhaseStartTest "test-$i"
     rlRun "fail"
     if fail:
       continue
     rlPhaseEnd
   done


Actual results:
[...]

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Satellite 5.4.1 i386 RHEL5 check
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

ls: Satellite-5.4.1-RHEL5-*: No such file or directory
:: [   FAIL   ] :: Not a directory: /i386/i386 

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Satellite 5.4.1 x86_64 RHEL5 check
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

[...]

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Satellite 5.4.1 i386 RHEL5 check
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [   FAIL   ] :: Not a directory: /i386/i386 
:: [   LOG    ] :: Duration: 18s
:: [   LOG    ] :: Assertions: 0 good, 1 bad
:: [unfinished] :: RESULT: Satellite 5.4.1 i386 RHEL5 check

[...]


Beaker WebUI or XML with results say "PASS" for the test.


Expected results:
Test should finish as FAIL as this is dangerous situation (gives false feeling all is OK, although there were FAIL). Even if no FAIL would be reported, IMO it is better to end with FAIL.
Comment 1 Petr Muller 2012-04-13 09:45:09 EDT
At first I wanted to immediately NOTABUG it, as I don't give much about the broken tests. But I guess it should not do any harm if we (in rlJournalEnd) checked if there's any unfinished phase, and issue a WARN about it. Tests not using rlJournalEnd are on their own, as there is no other place where such check could be placed.
Comment 2 Fedora End Of Life 2013-02-13 21:15:58 EST
Fedora 16 changed to end-of-life (EOL) status on 2013-02-12. Fedora 16 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.