Bug 702140 - Unable to determine pass/fail state of logs after packaging with v7 submit
Summary: Unable to determine pass/fail state of logs after packaging with v7 submit
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Hardware Certification Program
Classification: Retired
Component: Test Suite (harness)
Version: 6.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Greg Nichols
QA Contact: Guangze Bai
URL:
Whiteboard:
: 705871 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-05-04 21:08 UTC by Gregg Shick
Modified: 2015-02-08 21:36 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
In v7 1.3, the exit code after testing running was no longer printed which made it difficult to determine test PASS/FAIL state for user. This issue has been fixed in v7 1.4, now a PASS/FAIL string will be printed after each test execution finishes.
Clone Of:
Environment:
Last Closed: 2011-11-08 15:42:03 UTC


Attachments (Terms of Use)
report.py patch to add PASS/FAIL as last line of logs in results rpm (1.02 KB, patch)
2011-07-19 15:47 UTC, Greg Nichols
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2011:1436 0 normal SHIPPED_LIVE v7 bug fix and enhancement update 2011-11-08 23:28:41 UTC

Description Gregg Shick 2011-05-04 21:08:52 UTC
Description of problem:

The old kits ended with "exit code 0" in the logs before and after packaging to indicate a passing run.  We used this key when reviewing logs to ensure we were submitting passing results.  The new kit output.log files no longer has this after packaging.  This makes it more difficult to review logs that were run by external teams.

Is there something to key on in the logs to verify pass/fail status without submitting results to the website?

Version-Release number of selected component (if applicable):
v7-1-3-35 / 6.1 snap 5

How reproducible:
Everytime

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 Greg Nichols 2011-05-04 22:06:30 UTC
The v7 print command shows passing status of each test run.   The v7 server may also be run on the SUT, and test results formatted via web/css.

Comment 2 Gregg Shick 2011-05-04 22:08:11 UTC
Greg

What if we are not running the tests and have no access to the SUT at runtime?  We are only provided the final rpm for submission.

Comment 3 Rob Landry 2011-05-09 14:58:25 UTC
Hi Gregg,

Extracting results.xml from the rpm, and placing it locally will allow v7 print to work.  Alternatively the results.xml path can be set in /etc/v7.xml.

We can also look @ a 1.4 feature to enable using /var/v7/store directly in v7 which should simplify this and avoid needing a script.

Comment 4 Greg Nichols 2011-07-19 15:47:46 UTC
Created attachment 513831 [details]
report.py patch to add PASS/FAIL as last line of logs in results rpm

Comment 8 Greg Nichols 2011-09-19 13:17:35 UTC
*** Bug 705871 has been marked as a duplicate of this bug. ***

Comment 10 Caspar Zhang 2011-10-21 15:52:50 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
In v7 1.3, the exit code after testing running was no longer printed which made it difficult to determine test PASS/FAIL state for user. This issue has been fixed in v7 1.4, now a PASS/FAIL string will be printed after each test execution finishes.

Comment 11 errata-xmlrpc 2011-11-08 15:42:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1436.html

Comment 12 errata-xmlrpc 2011-11-08 18:30:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1436.html


Note You need to log in before you can comment on or make changes to this bug.