Bug 1299722 - Automate acceptance testing: automate provisioning
Automate acceptance testing: automate provisioning
Product: Beaker
Classification: Community
Component: tests (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified (vote)
: future_maint
: ---
Assigned To: beaker-dev-list
: FutureFeature
Depends On:
  Show dependency treegraph
Reported: 2016-01-19 01:15 EST by Roman Joost
Modified: 2017-07-26 00:58 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2017-07-26 00:58:02 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Roman Joost 2016-01-19 01:15:48 EST
Description of problem:

At this point in time testing provisioning is still a manual job when it comes to acceptance testing in Beaker. This backlog item is to create automated provisioning tests to avoid long manual testing for finding regressions.
Comment 1 Dan Callaghan 2017-07-26 00:58:02 EDT
I think this is essentially done, at this point. For bug 954265 I wrote bkr workflow-selftest, which produces a matrix of each supported combination (or least, the top tier most important) of distro and arch. The recipes include our existing modest selection of automated self-tests (currently just reboot and multihost sanity, see bug 961239 for expanding that set of tests).

The other piece of the puzzle is a Jenkins job to kick off the workflow and ensure that it comes back passing. We have that too, and have been using it the last few releases for automated acceptance testing:


Of course right now it always fails, because it's quite hard to get all our hardware to reliably boot and run. But the solution there is to just keep reducing the failure rate, whether it's fixing system configs, excluding unreliable distros/machines, or ultimately implementing automatic re-scheduling of aborted recipes:


Note You need to log in before you can comment on or make changes to this bug.