Red Hat Bugzilla – Bug 961239
Create a more comprehensive set of reference tasks to validate correct Beaker operation
Last modified: 2016-05-26 09:34:04 EDT
+++ This bug was initially created as a clone of Bug #954265 +++
Beaker's automated test suite is able to check that kickstarts for system provisioning are populated appropriately, but doesn't check that *task* environments are populated correctly by the harness. This has lead to a number of regressions over time (for example, in 0.12, duplicate entries in role variables, and potentially hanging in rhts-sync-block when using it between a host recipe and one of its guests). There are also some past (and existing) Beaker faults which can only be triggered by a misbehaving task, and are thus currently hard to test.
It would be beneficial to have a set of reference tasks and job specifications that can be run against a beaker instance to ensure additional things like:
1. System reboots work correctly
2. Extending the local watchdog works correctly
3. Extending the external watchdog works correctly
4. Reporting results and uploading logs works correctly
In addition to being useful in testing new Beaker versions, these reference tasks would be potentially useful for functional testing of new Beaker installations.
To help with integration testing of the main Beaker web UI, it would also be desirable to have tasks or job definitions that deliberately *fail* in certain ways:
1. Trigger a kernel panic
2. Run out the external watch dog
3. Run out the local watch dog
4. Report a warning
Also desirable for UI integration testing:
1. Large number of recipe sets in a job
2. Multi host recipe set
Also useful would be to check the handling of long task names and result paths.
Perhaps creating a simple "runscript" task would be possible, which accepted a shell script to run as CDATA in the task XML?
That would cover a lot of these test cases without a plethora of dedicated testing tasks.
Turns out the task we want (/distribution/command) already exists, we just need to publish and document it properly. Added bug 1018013 as a dependency.
Any new tasks we write here should be added to workflow-selftest: