Bug 810809

Summary: v7 tests result cannot be judged from its exit status
Product: [Retired] Red Hat Hardware Certification Program Reporter: Guangze Bai <gbai>
Component: Test Suite (harness)Assignee: Greg Nichols <gnichols>
Status: CLOSED ERRATA QA Contact: Zhiyou Liu <zhiliu>
Severity: low Docs Contact:
Priority: urgent    
Version: 1.5CC: czhang, rlandry, yshao
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
A bug fix has been added in v7 1.5 that now v7 commands now return non-zero value if the command execution fails.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2012-06-05 21:37:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 767775    
Attachments:
Description Flags
hardwaretest.py patch to return 1 if any test in the run fails none

Description Guangze Bai 2012-04-09 09:05:36 UTC 
Description of problem:

Running v7 tests with FAIL result, as we can see what v7 says like "Return value was 1", but its exit status gives zero. For instance,

# modprobe bfa
# v7 run -t info
...
Subtest: Kernel - Check OS kernel build, version
Error: Tech-preview modules not in whitelist:
bfa
Error: kernel is tainted (value = 536872960)
FAIL
...
Return value was 1
saved to /var/v7/results.xml
# echo $?
0                     <-- here it is

Version-Release number of selected component (if applicable):
v7-1.5-18

How reproducible:
always

Actual results:
$? always zero as long as tests end and not being terminated with abnormal errors such as backtrace.

Expected results:
$? represent various status: e.g.
0 - PASS
1 - normal FAIL
2 - abnormal FAIL (backtrace, ...)
Comment 1 Greg Nichols 2012-04-09 13:11:35 UTC 
Created attachment 576210 [details]
hardwaretest.py patch to return 1 if any test in the run fails
Comment 8 Caspar Zhang 2012-06-04 08:03:51 UTC 
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
A bug fix has been added in v7 1.5 that now v7 commands now return non-zero value if the command execution fails.
Comment 10 errata-xmlrpc 2012-06-05 21:37:34 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0711.html