Hide Forgot
Description of problem: Tests that require interaction currently are logged as passing, then marked as failing when the results rpm is built. Prior to building the results rpm, "v7 print" will show the test result as AUTO. The intent of --mode auto in v7 1.3 is to run tests in automated fashion. Tests fall into three catalogies, and should be treated accordingly: 1) interactive tests (example: cdrom, audio) These tests can not be run without user-interaction. These tests should be marked as failures if run in --mode auto 2) tests that have user prompts for usability/effectiveness, but can be run with out user intervention. (example: reboot) 3) non-interactive tests (info, core, etc.) --mode auto has no effect. Category 1) test code should be changed to fail the tests. Also, code in v7 to make interactive tests as failures during results submission (rpm or otherwise) should be removed. Version-Release number of selected component (if applicable): v7 1.3 R45
Created attachment 512264 [details] patch to fail interactive tests run in auto mode
(In reply to comment #0) > Also, code in v7 to > make interactive tests as failures during results submission (rpm or otherwise) > should be removed. This code seems not be removed yet in latest version?
(In reply to comment #4) > (In reply to comment #0) > > > Also, code in v7 to > > make interactive tests as failures during results submission (rpm or otherwise) > > should be removed. > > This code seems not be removed yet in latest version? The following code will never be reached: in reporttohtml.py: 159 if result == Constants.PASS and run.getMode() == Constants.auto and test.isInteractive(): 160 result = "auto" in report.py: 117 if test.isInteractive() and run.getMode() == Constants.auto and run.getSummary() == Constants.PASS: 118 summary = "- AUTO" Code still exist. Shall we fix it in 1.4 or defer the code cleaning to 1.5?
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: An enhancement has been made in v7 1.4 that when trying to run interactive tests with --mode auto option, the test will fail, instead of showing "AUTO" or being logged as "PASS".
(In reply to comment #6) > > Shall we fix it in 1.4 or defer the code cleaning to 1.5? defer.
(In reply to comment #8) > (In reply to comment #6) > > > > > Shall we fix it in 1.4 or defer the code cleaning to 1.5? > > defer. Bug 747990 reported for tracking code clean up process in 1.5.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2011-1436.html