Bug 961239 - Create a more comprehensive set of reference tasks to validate correct Beaker operation
Create a more comprehensive set of reference tasks to validate correct Beaker...
Status: NEW
Product: Beaker
Classification: Community
Component: tests (Show other bugs)
0.12
Unspecified Unspecified
high Severity unspecified (vote)
: future_maint
: ---
Assigned To: beaker-dev-list
tools-bugs
: FutureFeature
Depends On: 954265 1018013
Blocks:
  Show dependency treegraph
 
Reported: 2013-05-09 04:24 EDT by Nick Coghlan
Modified: 2016-05-26 09:34 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: 954265
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Nick Coghlan 2013-05-09 04:24:26 EDT
+++ This bug was initially created as a clone of Bug #954265 +++

Beaker's automated test suite is able to check that kickstarts for system provisioning are populated appropriately, but doesn't check that *task* environments are populated correctly by the harness. This has lead to a number of regressions over time (for example, in 0.12, duplicate entries in role variables, and potentially hanging in rhts-sync-block when using it between a host recipe and one of its guests). There are also some past (and existing) Beaker faults which can only be triggered by a misbehaving task, and are thus currently hard to test.

It would be beneficial to have a set of reference tasks and job specifications that can be run against a beaker instance to ensure additional things like:

1. System reboots work correctly
2. Extending the local watchdog works correctly
3. Extending the external watchdog works correctly
4. Reporting results and uploading logs works correctly

In addition to being useful in testing new Beaker versions, these reference tasks would be potentially useful for functional testing of new Beaker installations.
Comment 1 Nick Coghlan 2013-10-02 21:35:49 EDT
To help with integration testing of the main Beaker web UI, it would also be desirable to have tasks or job definitions that deliberately *fail* in certain ways:

1. Trigger a kernel panic
2. Run out the external watch dog
3. Run out the local watch dog
4. Report a warning

Also desirable for UI integration testing:
1. Large number of recipe sets in a job
2. Multi host recipe set
Comment 2 Nick Coghlan 2013-10-03 21:17:40 EDT
Also useful would be to check the handling of long task names and result paths.
Comment 3 Nick Coghlan 2013-10-10 20:58:34 EDT
Perhaps creating a simple "runscript" task would be possible, which accepted a shell script to run as CDATA in the task XML?

That would cover a lot of these test cases without a plethora of dedicated testing tasks.
Comment 4 Nick Coghlan 2013-10-10 22:22:56 EDT
Turns out the task we want (/distribution/command) already exists, we just need to publish and document it properly. Added bug 1018013 as a dependency.
Comment 5 Dan Callaghan 2016-04-06 00:59:29 EDT
Any new tasks we write here should be added to workflow-selftest:

https://git.beaker-project.org/cgit/beaker-workflow-selftest/

Note You need to log in before you can comment on or make changes to this bug.