Red Hat Bugzilla – Bug 972639
recipe_status from ditribution/reservesys has no output
Last modified: 2018-02-05 19:41:31 EST
Created attachment 759127 [details]
Description of problem:
recipe_status from ditribution/reservesys outputs only usage issues but provide no information about processing xmlrpc. Even when it exits with non-zero exit code it doesn`t print why.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. append to your favorite recipe
<task name="/distribution/reservesys" role="STANDALONE">
<param name="RESERVE_IF_FAIL" value="1"/>
<param name="RESERVETIME" value="6"/>
2. explore logs
no messages related to obtaining recipe status are present
log contains info about recipe status obtaining
obtained recipe status
info about exit status
small UNTESTED patch attached
It looks like /distribution/reservesys is working fine here. You passed RESERVETIME=6 which means the system is only reserved for 6 seconds. :-)
The next release of Beaker has many docs improvements, including a section about these "Beaker-provided" tasks which explains the RESERVETIME parameter. You can see the development version of the docs here:
OK, RESERVETIME was set to 6 seconds only, but RESERVE_IF_FAIL was set too.
It means that /distribution/reservesys should obtain current recipe status and if status is "pass" exit with pass status instead of reservation.
But reservesys continued as if recipe status is fail.
Created attachment 760483 [details]
RESERVETIME is *also* used to determine how long to extend the external watchdog when starting the reservesys task. The harness state updates and the XML-RPC call up to the proxy and hence back to the main Beaker server to determine the recipe result could easily reach the 6 seconds allocated here.
The reason there's no output from the reservesys task is because it didn't fail - it got aborted by the external watchdog and had its machine taken away (this is reported in the Beaker UI, not the logs).
We're going to provide a harness independent reservation mechanism that also works for alternative harnesses and tasks that have been aborted rather than failing (see bug 639938).
While the reservesys task *could* enforce a minimum time limit for the reservation, we're not inclined to tinker with it when we could put that time into the replacement mechanism is instead.
(In reply to Nick Coghlan from comment #6)
> The reason there's no output from the reservesys task
Petr's point in comment 5 is that there *is* output from the reservesys task and you can see that it did in fact try to reserve the system (though only for ~1 minute due to the RESERVETIME=6 parameter) even though the recipe had passed and so reservesys should have silently passed immediately.
The recipe_status script in reservesys will spew a traceback if something goes actually wrong while checking the recipe status. I think what is happening here is that the recipe status is not yet known at the time it checks, because of the delay from update_dirty_jobs (new in Beaker 0.12). There is a small window of time after the previous task finished in which the result would still stay at New. In that case the recipe_status script succeeds, but New != Pass so it is counted as a failure.
I think the right answer here is bug 639938 anyway, because it will difficult and awkward to handle the delayed result in the reservesys task itself.