Bug 961239

Summary:	Create a more comprehensive set of reference tasks to validate correct Beaker operation
Product:	[Retired] Beaker	Reporter:	Nick Coghlan <ncoghlan>
Component:	tests	Assignee:	beaker-dev-list
Status:	CLOSED WONTFIX	QA Contact:	tools-bugs <tools-bugs>
Severity:	unspecified	Docs Contact:
Priority:	high
Version:	0.12	CC:	jingwang, mastyk, qwan, tools-bugs, xtian
Target Milestone:	future_maint	Keywords:	FutureFeature
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:	954265	Environment:
Last Closed:	2020-06-02 09:55:52 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	954265, 1018013
Bug Blocks:

Description Nick Coghlan 2013-05-09 08:24:26 UTC

+++ This bug was initially created as a clone of Bug #954265 +++

Beaker's automated test suite is able to check that kickstarts for system provisioning are populated appropriately, but doesn't check that *task* environments are populated correctly by the harness. This has lead to a number of regressions over time (for example, in 0.12, duplicate entries in role variables, and potentially hanging in rhts-sync-block when using it between a host recipe and one of its guests). There are also some past (and existing) Beaker faults which can only be triggered by a misbehaving task, and are thus currently hard to test.

It would be beneficial to have a set of reference tasks and job specifications that can be run against a beaker instance to ensure additional things like:

1. System reboots work correctly
2. Extending the local watchdog works correctly
3. Extending the external watchdog works correctly
4. Reporting results and uploading logs works correctly

In addition to being useful in testing new Beaker versions, these reference tasks would be potentially useful for functional testing of new Beaker installations.

Comment 1 Nick Coghlan 2013-10-03 01:35:49 UTC

To help with integration testing of the main Beaker web UI, it would also be desirable to have tasks or job definitions that deliberately *fail* in certain ways:

1. Trigger a kernel panic
2. Run out the external watch dog
3. Run out the local watch dog
4. Report a warning

Also desirable for UI integration testing:
1. Large number of recipe sets in a job
2. Multi host recipe set

Comment 2 Nick Coghlan 2013-10-04 01:17:40 UTC

Also useful would be to check the handling of long task names and result paths.

Comment 3 Nick Coghlan 2013-10-11 00:58:34 UTC

Perhaps creating a simple "runscript" task would be possible, which accepted a shell script to run as CDATA in the task XML?

That would cover a lot of these test cases without a plethora of dedicated testing tasks.

Comment 4 Nick Coghlan 2013-10-11 02:22:56 UTC

Turns out the task we want (/distribution/command) already exists, we just need to publish and document it properly. Added bug 1018013 as a dependency.

Comment 5 Dan Callaghan 2016-04-06 04:59:29 UTC

Any new tasks we write here should be added to workflow-selftest:

https://git.beaker-project.org/cgit/beaker-workflow-selftest/