Bug 467737 - FEAT: hts certify command should recommend tests to complete certification
FEAT: hts certify command should recommend tests to complete certification
Status: CLOSED ERRATA
Product: Red Hat Hardware Certification Program
Classification: Red Hat
Component: Test Suite (harness) (Show other bugs)
5.2
All Linux
medium Severity medium
: ---
: ---
Assigned To: Greg Nichols
Lawrence Lim
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-10-20 12:18 EDT by Greg Nichols
Modified: 2014-03-25 20:55 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-01-27 17:58:03 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
new ResultsEngine class for assessing test results and recomending test runs (2.60 KB, text/x-python)
2008-10-20 12:25 EDT, Greg Nichols
no flags Details
hardwaretest.py patch to change certify command to use ResultsEngine class (2.50 KB, patch)
2008-10-20 12:37 EDT, Greg Nichols
no flags Details | Diff
test.py patch to add mandatory attribute (333 bytes, patch)
2008-10-20 12:38 EDT, Greg Nichols
no flags Details | Diff
certificationtest.py patch to add mandatory test attribute (751 bytes, patch)
2008-10-20 12:57 EDT, Greg Nichols
no flags Details | Diff
tag.py patch to add necessary XML entity and attribute names. (263 bytes, patch)
2008-10-20 13:02 EDT, Greg Nichols
no flags Details | Diff
suspend.py patch to designate it as mandatory, pre (247 bytes, patch)
2008-10-20 13:05 EDT, Greg Nichols
no flags Details | Diff
info.py patch to designate the test as mandatory post (253 bytes, patch)
2008-10-20 13:06 EDT, Greg Nichols
no flags Details | Diff

  None (edit)
Description Greg Nichols 2008-10-20 12:18:57 EDT
Description of problem:

The "hts certify" command should be revised to consider existing test results and recommend only those tests required for certification.  For example, if the memory test has already passed, it should conduct a test run without the memory test.

A new command, "hts run" could preserve the functionality of the existing "hts certify" command, allowing a manually-defined test run.
Comment 1 Greg Nichols 2008-10-20 12:25:26 EDT
Created attachment 320880 [details]
new ResultsEngine class for assessing test results and recomending test runs


The ResultsEngine class analyses test runs from the HTS database (/var/hts/results.xml) and makes recommendations for additional testing.

This initial implementation evaluates test runs to be sure that any mandatory
tests have passed in the same run.  If so, the test may be omitted from the
recommended test run.
Comment 2 Greg Nichols 2008-10-20 12:37:39 EDT
Created attachment 320885 [details]
hardwaretest.py patch to change certify command to use ResultsEngine class


This patch changes HTS, redefining "hts certify" to use the new ResultsEngine, and adding the "hts run" command as the manual test run command.

The patch also uses the new "mandatory" test designations to determine what tests are required every run, and whether they should preceed or follow the requested tests.
Comment 3 Greg Nichols 2008-10-20 12:38:48 EDT
Created attachment 320886 [details]
test.py patch to add mandatory attribute
Comment 4 Greg Nichols 2008-10-20 12:57:11 EDT
Created attachment 320890 [details]
certificationtest.py patch to add mandatory test attribute


This patch adds the mandatory attribute to tests.
Comment 5 Greg Nichols 2008-10-20 13:02:44 EDT
Created attachment 320891 [details]
tag.py patch to add necessary XML entity and attribute names. 


Add necessary XML entity and attribute names for mandatory attributes.
Comment 6 Greg Nichols 2008-10-20 13:05:09 EDT
Created attachment 320893 [details]
suspend.py patch to designate it as mandatory, pre
Comment 7 Greg Nichols 2008-10-20 13:06:27 EDT
Created attachment 320894 [details]
info.py patch to designate the test as mandatory post
Comment 10 Yan Tian 2009-01-01 23:44:58 EST
Verified "hts certify" will execute all tests which not passed and "hts run" could execute with specified test or device in hts-5.3.12.
Comment 12 errata-xmlrpc 2009-01-27 17:58:03 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-0047.html

Note You need to log in before you can comment on or make changes to this bug.