Bug 623933 - RFE: add reservesys when previous tests fail
Summary: RFE: add reservesys when previous tests fail
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Beaker
Classification: Retired
Component: lab controller
Version: 0.5
Hardware: All
OS: Linux
high
medium
Target Milestone: ---
Assignee: Bill Peck
QA Contact:
URL:
Whiteboard:
: 596912 (view as bug list)
Depends On:
Blocks: 545868 853383
TreeView+ depends on / blocked
 
Reported: 2010-08-13 07:53 UTC by Petr Sklenar
Modified: 2019-05-22 13:42 UTC (History)
14 users (show)

Fixed In Version:
Clone Of:
: 853383 (view as bug list)
Environment:
Last Closed: 2012-09-06 14:55:26 UTC
Embargoed:


Attachments (Terms of Use)

Description Petr Sklenar 2010-08-13 07:53:04 UTC
Description of problem:
When we are fixing the test or fixing testing package then we need test /distribution/reservesys after each recipe. If scheduler could reserve machine for some time when previous test fails it would spare machine usage.

Version-Release number of selected component (if applicable):
https://beaker.engineering.redhat.com
Version - 0.5.53 

How reproducible:
always

Steps to Reproduce:
1. test A
2. test B
3. all other tests
last test: 4. when there were fail ^ schedule /distribition/reservesys
ELSE
nothing
  
Actual results:
nothing like that, each test which needs to be investigated has a reservesys

Expected results:


Additional info:
It could spare more scheduling of machines

Comment 2 Marian Csontos 2010-08-13 08:50:56 UTC
Thanks for a good idea.

Though I am not sure we can safely insert tasks into recipe as it could break multihost tests, this would not be difficult to implement in harness:

Simple on failure - extend watchdog and notify owner.

Comment 3 Šimon Lukašík 2010-08-13 09:23:00 UTC
I would really like to see this implemented. I think this feature when used with wisdom, could save some machine-time and man-time. For instance:

 - If you don't use reservesys and job fails, you need to wait for new machine to check/repair. So you wait for provisioning and test end. Note that some of the tests may last hours. It's also possible, that problem is not deterministic and second job will pass.

 - If you use reservesys and job didn't fail, you need to return each machine which you reserved. Is this exactly what people often forget?

It could be also implemented if reservesys would have some option like:
--task=/distribution/reservesys --taskparam="RETURN_WHEN_PASS=True"

Comment 4 Marian Csontos 2010-08-13 09:34:30 UTC
That's approximately what I am thinking about.

And it should also report results of manual runs :-)

task
|| /task | Fail ||
|| /reserve | Pass ||
|| /manual-run/task | Fail ||
|| /manual-run/task | Pass ||
|| /reserve/end | Pass ||
next_task
...

(unless one would want not to...)

Comment 5 Šimon Lukašík 2010-08-13 09:45:59 UTC
(In reply to comment #4)
> task
> || /task | Fail ||
> || /reserve | Pass ||
> || /manual-run/task | Fail ||
> || /manual-run/task | Pass ||
> || /reserve/end | Pass ||
> next_task
> ...

+1

Comment 6 David Kovalsky 2011-04-11 10:43:05 UTC
Caution has to be taken so all machines in Beaker are not blocked out. If a mass run on limited architectures (think s390s) fails and machine gets reserved for extra 24h (think - Friday evening) then this doesn't help you much and still holds the machine. See also bug 694733.

I guess we can use common sense and educate people :)

Comment 7 Šimon Lukašík 2011-04-11 10:59:22 UTC
David, I completely agree with common sense. :) 

Still, I think it worth to consider at least something like:

  --task=/distribution/reservesys --taskparam="RETURN_WHEN_PASS=True"

Such mechanism, I hope, when used instead of reservesys could save
some machine time.

Comment 8 Martin Cermak 2012-01-18 15:46:30 UTC
Very good idea, +1!

The risk of machine block-out is serious however. Maybe some very short reserve time could be used by default and the user would have to run extendtestingtime.sh manually if he really planned to dig into the problem.

Comment 9 Bill Peck 2012-08-20 17:01:01 UTC
on gerrit for review

Comment 13 Bill Peck 2012-08-30 14:10:43 UTC
Hello Petr,

Here is the syntax to use for this

<task name="/distribution/reservesys" role="STANDALONE">
	<params>
		<param name="RESERVETIME" value="86400"/>
		<param name="RESERVE_IF_FAIL" value="1"/>
	</params>
</task>

Comment 14 Dan Callaghan 2012-08-30 23:12:38 UTC
(In reply to comment #12)
> any previous test failed then reservesys?

Yes, this one. Pass RESERVE_IF_FAIL="1" to reservesys as Bill has shown, and then the reservation will only happen if some other task in the recipe has failed. If everything passed then /distribution/reservesys will also immediately pass without reserving the system.

Comment 15 Dan Callaghan 2012-09-06 04:39:18 UTC
Beaker 0.9.3 has been released.

Comment 19 Dan Callaghan 2012-09-28 01:32:14 UTC
*** Bug 596912 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.