It would be useful if beaker could support a SKIP result in addition to PASS, FAIL, and WARN. This would help clearly identify test cases where the case is "skipped" because of outside factors. Allowing a test to set a result to SKIP would clearly tell the tester or anyone viewing the results that a test didn't just fail or pass but, it was skipped.
Do the progress bars need to show completed tasks with Skip result? Would be confused to show them as completed tasks with no result?
Yes I think we have to include them in the progress bar like all other tasks. I guess they would be coloured the same as New result (plain grey).
I would expect a different color than New just to know that something isn't still expected to run. Right now the bar uses orange for warning right? Would yellow for skipped be too close to that? Or maybe purple? I think that's what Polarion uses for blocked tests. I'm not sure how universal that is.
Currently we show completed tasks with None or New result in the progress bar as plain grey. To use a different colour for skipped tests, I guess we need to add a new column 'stasks' in Job, Recipe Set and Recipe table to record how many tests have been skipped in a job. Not sure which colour is the best one for this. May be white which indicates nothing has been run?
Using white would work, I think.
How is the progress here? We have depending bug on beakerlib too.
Matt is on PTO so I can take this. Purple works. We can't use white, it would be indistinguishable from the background of the progress bar (which is what shows incomplete tasks).
Should be very similar to the change for adding None result in bug 915128 (which incidentally, does not appear to be properly supported in the web UI right now...) https://git.beaker-project.org/cgit/beaker/commit/?id=67cf7f9688272d4406a41254857768a239287f0f
This is a patch for the Beaker side of things: http://gerrit.beaker-project.org/5023 Skip will only be visible properly on the new job page (23.0+). The old job page will not show Skipped tasks in the progress bar. Some more will be needed to make rhts and beah aware of the new result type. I *think* restraint already supports it though.
Created attachment 1176737 [details] example screenshot showing Skip result
rhts: http://gerrit.beaker-project.org/5030 beah: http://gerrit.beaker-project.org/5029
Plus a Beaker task to exercise the new result type: http://gerrit.beaker-project.org/5031 and a patch to include it in bkr workflow-selftest: http://gerrit.beaker-project.org/5032
Dan, Will that also work at the phase level? Thanks, Scott
(In reply to Scott Poore from comment #13) > Will that also work at the phase level? Not sure what you mean "at the phase level". Beaker doesn't know anything about phases, I guess that's a question for the beakerlib devs? What this bug gives is the ability to do: rhts-report-result $TEST SKIP 0
Ok. I think this is what I'm looking for. I'll see how the beakerlib bug/rfe comes along for this. Thanks.
Beaker 24.0 has been released.