Bug 842794 - [RFE] more resilient tests
[RFE] more resilient tests
Product: Fedora
Classification: Fedora
Component: beakerlib (Show other bugs)
All Linux
medium Severity medium
: ---
: ---
Assigned To: Petr Muller
: 667427 980128 (view as bug list)
Depends On:
Blocks: Fedora-beakerlib-1.9-3 1116317
  Show dependency treegraph
Reported: 2012-07-24 11:46 EDT by Jiri Jaburek
Modified: 2016-09-19 22:09 EDT (History)
7 users (show)

See Also:
Fixed In Version: beakerlib-1.9-3.fc20
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2014-07-30 02:57:41 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Jiri Jaburek 2012-07-24 11:46:57 EDT
The typical scenario for a current beakerlib-using tests is something like

# includes here
. /usr/bin/rhts-environment.sh
. /usr/share/beakerlib/beakerlib.sh


Note that this structure/model is purely visual, all rl* functions use here do logging only, none of them cares about the actual setup/test/cleanup state. That's not a problem for simple tests (ie. reproducer wrappers).

However when a test needs to perform destructive actions (backed by rlFileBackup, restored using rlFileRestore), this flow is not good enough. If such test is interrupted during the Test phase, no cleanup is performed and thus no rlFileRestore is called, causing the OS to become dirty (ie. custom test-provided /etc/passwd is not replaced by the original one). This interrupt can be user-triggered (ie. SIGINT / ctrl-c) or timer-triggered (ie. SIGKILL sent by beaker daemon when local watchdog expires).
And since the test script itself (ie. runtest.sh) is called directly from Makefile, there's noone left to run the cleanup.

A test author is able to trap those signals and write custom cleanup functions (called in place of the current cleanup phase), but he/she shouldn't have to.

My proposal (RFE) is to run the runtest.sh script a wrapper, which would trap the SIGTERM signal (sent by beaker on localwatchdog expire), SIGKILL the runtest.sh (run as child of this wrapper) and all its children, and execute commands stored in a "cleanup buffer".
The cleanup buffer could be a file (set of files) or a variable, filled during runtest.sh execution by the test author using something like rlCleanupAppend and rlCleanupPrepend. Those functions would append a command to the cleanup buffer, or prepend the command before everything else in the buffer, respectively. This cleanup buffer would be thus executed upon runtest.sh termination.

This wrapper would be fully backwards-compatible. There's no need for the test author to use rlCleanup* functions, new tests could be still made using the original syntax. Existing tests would not be affected, due to their Makefiles executing runtest.sh directly, though using the wrapper in such case would still bring no harm - if the cleanup buffer is empty, nothing after the runtest.sh is executed.

The rlPhaseStartCleanup phase in new tests could completely empty (or not specified at all), a cleanup phase with different name would be used for the cleanup buffer.

Example follows:

# includes here
. /usr/bin/rhts-environment.sh
. /usr/share/beakerlib/beakerlib.sh

        rlFileBackup /etc/passwd
        rlFileBackup /etc/group
        rlFileBackup /etc/shadow
        rlCleanupAppend "rlFileRestore"
        rlRun "useradd joe"
        rlRun "echo thisisasimplepassword | passwd --stdin root"
        rlCleanupPrepend "rm -rf /usr/local/example_testing"
        rlRun "mkdir /usr/local/example_testing"
        rlRun "cp -v /etc/yum.repos.d/* /usr/local/example_testing"
        rlRun "sleep 6000000"

When this test is run from a terminal (manually) and a SIGINT is sent (^C), rlFileRestore is called and the original environment is restored. Same applies to beaker, given ie. "10m" runtime (in Makefile). As the cleanup buffer is executed, all commands in it are logged in a style similar (or same as) rlRun.

Use cases for this feature are quite obvious - to perform cleanup when the test is interrupted or even SIGKILLed. Several people have been asking me for such feature and I personally would welcome it as well, given the "destructive" nature (in using file backup+restore) of the tests most of us need to write quite often.
Comment 2 Petr Muller 2013-06-28 10:29:25 EDT
I believe Jiri has provided some patches for this RFE, and posted them to beaker list. I plan to review them sometimes soon
Comment 3 Petr Muller 2013-06-28 11:14:47 EDT
*** Bug 667427 has been marked as a duplicate of this bug. ***
Comment 4 Petr Muller 2013-07-01 10:19:04 EDT
*** Bug 980128 has been marked as a duplicate of this bug. ***
Comment 5 Dalibor Pospíšil 2013-07-08 04:55:57 EDT
I have just moved my functions of this as beakerlib library distribution/Cleanup.

It can be found here https://tcms.engineering.redhat.com/case/287558.

This library does not solve problem with local watchdog but it could be extended. It can handle ^C while executing manually. It does not allow partial cleanup. If there is a demand it could be implemented.
Comment 8 Fedora Update System 2014-06-17 09:42:21 EDT
beakerlib-1.9-1.fc20 has been submitted as an update for Fedora 20.
Comment 9 Fedora Update System 2014-06-17 19:29:50 EDT
Package beakerlib-1.9-1.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing beakerlib-1.9-1.fc20'
as soon as you are able to.
Please go to the following url:
then log in and leave karma (feedback).
Comment 10 Fedora Update System 2014-07-02 04:02:02 EDT
beakerlib-1.9-2.fc20 has been submitted as an update for Fedora 20.
Comment 11 Dalibor Pospíšil 2014-07-04 04:50:58 EDT
Please consider the package fixing this bug available in Fedora stable repos once bz1116308 is closed and RHEL stable repos once bz1116317 is closed.

Fixed in:
Comment 12 Fedora Update System 2014-07-17 09:16:07 EDT
beakerlib-1.9-3.fc20 has been submitted as an update for Fedora 20.
Comment 13 Fedora Update System 2014-07-30 02:57:41 EDT
beakerlib-1.9-3.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.