Description of problem:
beaker-watchdog is aborting recipes too early.
Version-Release number of selected component (if applicable):
0.17.1
Steps to Reproduce:
currently unknown
Additional info:
Have not confirmed this yet, but from looking at this commit:
https://git.beaker-project.org/cgit/beaker/commit/Server/bkr/server/model/scheduler.py?id=525ef3e29d83dd2dcdba24facd0669826ae49912
it seems like Beaker will return a watchdog as 'expired' if *any* recipe in the set has kill time in the past. It should be only if *all* recipes in the set have kill time in the past. However, this doesn't yet fully explain the behaviour we are seeing.
I noticed that the guest recipes are always aborted exactly 4 hours after the host recipe finished /distribution/virt/start, I feel like that must be a clue as to what exactly is going wrong...
Okay so the mystery of the 4 hours is solved: beah extends the watchdog by 4 hours when a task finishes (I don't know what for).
The issue here is really just with the Watchdog.by_status query.
Steps to reproduce:
1. Submit a job with a host recipe containing a guest recipe, with the following tasks:
Host
/distribution/install
/distribution/virt/install
/distribution/virt/start
Guest
/distribution/install
/distribution/reservesys
Actual results:
Job runs up until reservesys in the guest, but instead of staying reserved for 24 hours the job is terminated by external watchdog 4 hours after /distribution/virt/start completed.
Expected results:
Job runs successfully. reservesys task in the guest reserves for 24 hours.