Bug 618528

Summary:	Test watchdog does not work properly (coming tests might be affected by previous timeout)
Product:	[Retired] Beaker	Reporter:	Frantisek Reznicek <freznice>
Component:	beah	Assignee:	Bill Peck <bpeck>
Status:	CLOSED DUPLICATE	QA Contact:
Severity:	high	Docs Contact:
Priority:	high
Version:	0.5	CC:	bpeck, esammons, kbaker, mcsontos, rmancy
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2010-07-27 10:57:50 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Frantisek Reznicek 2010-07-27 08:04:51 UTC

Description of problem:

I have experienced multiple issues with watchdog here:
https://beaker.engineering.redhat.com/jobs/8851
https://beaker.engineering.redhat.com/jobs/8798
https://beaker.engineering.redhat.com/jobs/8801

The behavior is following, if a test is timeouting and test maximum duration is reached, then the test might end up with Warn status and all tests which are scheduled after this test also timeout, BUT with the watchdog time of the first timeouted test (2010-07-26 12:38:28).
It is clealy visible here: https://beaker.engineering.redhat.com/jobs/8851

Moreover there is possibility that test timeouts with a lot of "./" lines in WebUI which is quite interesting behavior and I tend to claim that this is invalid test watchdog trigger.

Let's closely look at
https://beaker.engineering.redhat.com/recipes/16074#task199644
(/distribution/MRG/Messaging/qpid_test_rhm_docs_examples test)

The test is set to use 1h watchdog timeout, but as you can see in the job/recipe it took 4:45:50 insted of 1:00:00 +- a minute.

Version-Release number of selected component (if applicable):
0.5.50 (WebUI version)

How reproducible:
quite frequently

Steps to Reproduce:
1. look at https://beaker.engineering.redhat.com/jobs/8851 or clone that test
2. see the timeouted test after first timeouting test
  
Actual results:
Beaker test watchdog is not behaving properly which results in test data loss.

Expected results:
Beaker test watchdog should behave properly.

Additional info:

Comment 1 Marian Csontos 2010-07-27 10:57:50 UTC

The long timeout is caused by harness allowing longer timeout to submit results to LC. This is to avoid watchdog killing the job when the task is already finished and harness is waiting for LC to submit results.

For the rest see Bug 618123.

*** This bug has been marked as a duplicate of bug 618123 ***