Red Hat Bugzilla – Bug 842794
[RFE] more resilient tests
Last modified: 2016-09-19 22:09:13 EDT
The typical scenario for a current beakerlib-using tests is something like
# includes here
Note that this structure/model is purely visual, all rl* functions use here do logging only, none of them cares about the actual setup/test/cleanup state. That's not a problem for simple tests (ie. reproducer wrappers).
However when a test needs to perform destructive actions (backed by rlFileBackup, restored using rlFileRestore), this flow is not good enough. If such test is interrupted during the Test phase, no cleanup is performed and thus no rlFileRestore is called, causing the OS to become dirty (ie. custom test-provided /etc/passwd is not replaced by the original one). This interrupt can be user-triggered (ie. SIGINT / ctrl-c) or timer-triggered (ie. SIGKILL sent by beaker daemon when local watchdog expires).
And since the test script itself (ie. runtest.sh) is called directly from Makefile, there's noone left to run the cleanup.
A test author is able to trap those signals and write custom cleanup functions (called in place of the current cleanup phase), but he/she shouldn't have to.
My proposal (RFE) is to run the runtest.sh script a wrapper, which would trap the SIGTERM signal (sent by beaker on localwatchdog expire), SIGKILL the runtest.sh (run as child of this wrapper) and all its children, and execute commands stored in a "cleanup buffer".
The cleanup buffer could be a file (set of files) or a variable, filled during runtest.sh execution by the test author using something like rlCleanupAppend and rlCleanupPrepend. Those functions would append a command to the cleanup buffer, or prepend the command before everything else in the buffer, respectively. This cleanup buffer would be thus executed upon runtest.sh termination.
This wrapper would be fully backwards-compatible. There's no need for the test author to use rlCleanup* functions, new tests could be still made using the original syntax. Existing tests would not be affected, due to their Makefiles executing runtest.sh directly, though using the wrapper in such case would still bring no harm - if the cleanup buffer is empty, nothing after the runtest.sh is executed.
The rlPhaseStartCleanup phase in new tests could completely empty (or not specified at all), a cleanup phase with different name would be used for the cleanup buffer.
# includes here
rlRun "useradd joe"
rlRun "echo thisisasimplepassword | passwd --stdin root"
rlCleanupPrepend "rm -rf /usr/local/example_testing"
rlRun "mkdir /usr/local/example_testing"
rlRun "cp -v /etc/yum.repos.d/* /usr/local/example_testing"
rlRun "sleep 6000000"
When this test is run from a terminal (manually) and a SIGINT is sent (^C), rlFileRestore is called and the original environment is restored. Same applies to beaker, given ie. "10m" runtime (in Makefile). As the cleanup buffer is executed, all commands in it are logged in a style similar (or same as) rlRun.
Use cases for this feature are quite obvious - to perform cleanup when the test is interrupted or even SIGKILLed. Several people have been asking me for such feature and I personally would welcome it as well, given the "destructive" nature (in using file backup+restore) of the tests most of us need to write quite often.
I believe Jiri has provided some patches for this RFE, and posted them to beaker list. I plan to review them sometimes soon
*** Bug 667427 has been marked as a duplicate of this bug. ***
*** Bug 980128 has been marked as a duplicate of this bug. ***
I have just moved my functions of this as beakerlib library distribution/Cleanup.
It can be found here https://tcms.engineering.redhat.com/case/287558.
This library does not solve problem with local watchdog but it could be extended. It can handle ^C while executing manually. It does not allow partial cleanup. If there is a demand it could be implemented.
beakerlib-1.9-1.fc20 has been submitted as an update for Fedora 20.
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing beakerlib-1.9-1.fc20'
as soon as you are able to.
Please go to the following url:
then log in and leave karma (feedback).
beakerlib-1.9-2.fc20 has been submitted as an update for Fedora 20.
Please consider the package fixing this bug available in Fedora stable repos once bz1116308 is closed and RHEL stable repos once bz1116317 is closed.
beakerlib-1.9-3.fc20 has been submitted as an update for Fedora 20.
beakerlib-1.9-3.fc20 has been pushed to the Fedora 20 stable repository. If problems still persist, please make note of it in this bug report.