Currently, the only reference to the ack/nak/comment system for recipe results appears to be a brief one in the capabilities list in the architecture guide: https://beaker-project.org/docs/architecture-guide/capabilities.html#results It's highly desirable to get that clearly documented, so we can use it as a basis for evolving the feature in the future. In particular, bug 1013445 (which covers a page redesign for the main results page) should be informed by a clear understanding of the intended purpose of this capability. It's also relevant to improving the integration with the jobs matrix (i.e. if we implement bug 641219 to hide nak'ed results by default, we will need to link to docs that explain what nak'ing a result means). Bug 620968 (which would allow filteringout nak'ed jobs from the main results) would also benefit from having docs to reference.
Oops, that matrix view issue is bug 649219.
Note, I think it's more important to get this covered in the Architecture Guide than it is in the user guide. The architecture guide write-up is what we need as a basis for future design changes, and to explain to users that are curious the "why" of the system. For actual *usage*, we should be aiming for it to be sufficiently intuitive in the web UI that the users don't need dedicated docs.
I'm not sure what exactly we need to add to the architecture guide here... It has this para currently: "To help filter out failures that are due to external environment issues (such as an outage of the lab network), results supports an acknowledgement system, where results may be accepted or rejected to indicate whether or not they represent a true failure or an incidental failure not related to the specific components being tested." which pretty much describes the entire feature and its purpose. I guess I can flesh that out into its own section "Result acknowledgement" which just mentions that: * it's per recipe-set * it's optional (defaults to neither ack nor nack) * you can add a comment as well
Yep, that's what I had in mind - more details about what's already possible. If there's an API for it, then that may of interest to the virtlab folks, if not, then we should consider adding one (perhaps as part of a larger revamp of the results page).
Original design of the feature was done in bug 596410.
Description of the review system: http://gerrit.beaker-project.org/3307 XMLRPC API docs for setting response (note getting/setting comment is currently not covered): http://gerrit.beaker-project.org/3306 bkr job-modify --response is already documented: https://beaker-project.org/docs/man/bkr-job-modify.html#cmdoption-bkr-job-modify--response
(In reply to Dan Callaghan from comment #7) > XMLRPC API docs for setting response (note getting/setting comment is > currently not covered): I mean, not covered by the API. There's no XMLRPC method to do it.
This bug fix is available on the Beaker web site: https://beaker-project.org/docs-release-0.18/server-api/xmlrpc.html#bkr.server.jobs.jobs.set_response https://beaker-project.org/docs-release-0.18/architecture-guide/capabilities.html#result-comments-and-waivers
Beaker 0.18.1 has been released.