This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 842794 - [RFE] more resilient tests
[RFE] more resilient tests
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: beakerlib (Show other bugs)
rawhide
All Linux
medium Severity medium
: ---
: ---
Assigned To: Petr Muller
:
: 667427 980128 (view as bug list)
Depends On:
Blocks: Fedora-beakerlib-1.9-3 1116317
  Show dependency treegraph
 
Reported: 2012-07-24 11:46 EDT by Jiri Jaburek
Modified: 2016-09-19 22:09 EDT (History)
7 users (show)

See Also:
Fixed In Version: beakerlib-1.9-3.fc20
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-07-30 02:57:41 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jiri Jaburek 2012-07-24 11:46:57 EDT
The typical scenario for a current beakerlib-using tests is something like

# BEGIN
# includes here
. /usr/bin/rhts-environment.sh
. /usr/share/beakerlib/beakerlib.sh

rlJournalStart
    rlPhaseStartSetup
        ...
    rlPhaseEnd
    rlPhaseStartTest
        ...
    rlPhaseEnd
    rlPhaseStartCleanup
        ...
    rlPhaseEnd
rlJournalPrintText
rlJournalEnd
# END

Note that this structure/model is purely visual, all rl* functions use here do logging only, none of them cares about the actual setup/test/cleanup state. That's not a problem for simple tests (ie. reproducer wrappers).

However when a test needs to perform destructive actions (backed by rlFileBackup, restored using rlFileRestore), this flow is not good enough. If such test is interrupted during the Test phase, no cleanup is performed and thus no rlFileRestore is called, causing the OS to become dirty (ie. custom test-provided /etc/passwd is not replaced by the original one). This interrupt can be user-triggered (ie. SIGINT / ctrl-c) or timer-triggered (ie. SIGKILL sent by beaker daemon when local watchdog expires).
And since the test script itself (ie. runtest.sh) is called directly from Makefile, there's noone left to run the cleanup.

A test author is able to trap those signals and write custom cleanup functions (called in place of the current cleanup phase), but he/she shouldn't have to.

My proposal (RFE) is to run the runtest.sh script a wrapper, which would trap the SIGTERM signal (sent by beaker on localwatchdog expire), SIGKILL the runtest.sh (run as child of this wrapper) and all its children, and execute commands stored in a "cleanup buffer".
The cleanup buffer could be a file (set of files) or a variable, filled during runtest.sh execution by the test author using something like rlCleanupAppend and rlCleanupPrepend. Those functions would append a command to the cleanup buffer, or prepend the command before everything else in the buffer, respectively. This cleanup buffer would be thus executed upon runtest.sh termination.

This wrapper would be fully backwards-compatible. There's no need for the test author to use rlCleanup* functions, new tests could be still made using the original syntax. Existing tests would not be affected, due to their Makefiles executing runtest.sh directly, though using the wrapper in such case would still bring no harm - if the cleanup buffer is empty, nothing after the runtest.sh is executed.

The rlPhaseStartCleanup phase in new tests could completely empty (or not specified at all), a cleanup phase with different name would be used for the cleanup buffer.


Example follows:

# BEGIN
# includes here
. /usr/bin/rhts-environment.sh
. /usr/share/beakerlib/beakerlib.sh

rlJournalStart
    rlPhaseStartSetup
        rlFileBackup /etc/passwd
        rlFileBackup /etc/group
        rlFileBackup /etc/shadow
        rlCleanupAppend "rlFileRestore"
    rlPhaseEnd
    rlPhaseStartTest
        rlRun "useradd joe"
        rlRun "echo thisisasimplepassword | passwd --stdin root"
        rlCleanupPrepend "rm -rf /usr/local/example_testing"
        rlRun "mkdir /usr/local/example_testing"
        rlRun "cp -v /etc/yum.repos.d/* /usr/local/example_testing"
        rlRun "sleep 6000000"
    rlPhaseEnd
    rlPhaseStartCleanup
    rlPhaseEnd
rlJournalPrintText
rlJournalEnd
# END

When this test is run from a terminal (manually) and a SIGINT is sent (^C), rlFileRestore is called and the original environment is restored. Same applies to beaker, given ie. "10m" runtime (in Makefile). As the cleanup buffer is executed, all commands in it are logged in a style similar (or same as) rlRun.

Use cases for this feature are quite obvious - to perform cleanup when the test is interrupted or even SIGKILLed. Several people have been asking me for such feature and I personally would welcome it as well, given the "destructive" nature (in using file backup+restore) of the tests most of us need to write quite often.
Comment 2 Petr Muller 2013-06-28 10:29:25 EDT
I believe Jiri has provided some patches for this RFE, and posted them to beaker list. I plan to review them sometimes soon
Comment 3 Petr Muller 2013-06-28 11:14:47 EDT
*** Bug 667427 has been marked as a duplicate of this bug. ***
Comment 4 Petr Muller 2013-07-01 10:19:04 EDT
*** Bug 980128 has been marked as a duplicate of this bug. ***
Comment 5 Dalibor Pospíšil 2013-07-08 04:55:57 EDT
I have just moved my functions of this as beakerlib library distribution/Cleanup.

It can be found here https://tcms.engineering.redhat.com/case/287558.

This library does not solve problem with local watchdog but it could be extended. It can handle ^C while executing manually. It does not allow partial cleanup. If there is a demand it could be implemented.
Comment 8 Fedora Update System 2014-06-17 09:42:21 EDT
beakerlib-1.9-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/beakerlib-1.9-1.fc20
Comment 9 Fedora Update System 2014-06-17 19:29:50 EDT
Package beakerlib-1.9-1.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing beakerlib-1.9-1.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-7442/beakerlib-1.9-1.fc20
then log in and leave karma (feedback).
Comment 10 Fedora Update System 2014-07-02 04:02:02 EDT
beakerlib-1.9-2.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/beakerlib-1.9-2.fc20
Comment 11 Dalibor Pospíšil 2014-07-04 04:50:58 EDT
Please consider the package fixing this bug available in Fedora stable repos once bz1116308 is closed and RHEL stable repos once bz1116317 is closed.

Fixed in:
beakerlib-1.9-2.fc19
beakerlib-1.9-2.fc20
beakerlib-1.9-2.fc21
beakerlib-1.9-3.el5
beakerlib-1.9-2.el6
beakerlib-1.9-2.el7
Comment 12 Fedora Update System 2014-07-17 09:16:07 EDT
beakerlib-1.9-3.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/beakerlib-1.9-3.fc20
Comment 13 Fedora Update System 2014-07-30 02:57:41 EDT
beakerlib-1.9-3.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.