Bug 503940 - [RFE] Add multihost test support
Summary: [RFE] Add multihost test support
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: beakerlib
Version: rawhide
Hardware: All
OS: Linux
low
low
Target Milestone: ---
Assignee: Dalibor Pospíšil
QA Contact:
URL:
Whiteboard: Project: Medium
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-06-03 13:41 UTC by Petr Šplíchal
Modified: 2016-09-20 02:12 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Petr Šplíchal 2009-06-03 13:41:57 UTC
When writing a multihost test, quite a bit of the code is always
the same:

    * find out if running in debugging mode or on RHTS
    * optional manual setting of SERVERS & CLIENTS variables
    * deciding whether we are on a server or a client

I suggest to implement a rlMultihost (or rlRunMultihost?) function
that would incorporate all these duplicating steps.

Suggested syntax: 

    rlMultihost Server Client [ServerHost ClientHost]

    * Server, Client - functions providing respective actions
    * ServerHost, ClientHost - hostname to be set in devel mode

The test should fail when hostname is not present in any of
CLIENTS and SERVERS variables, something like this:

    Fail() {
        rlPhaseStartTest Stray
            rlFail "Not a server, client neither. What shall I do?"
            rlLog "Run the test using the multihost-workflow"
            rlLog "Set correct hostnames when run manually"
        rlPhaseEnd
    }

The core of the rlMultihost does the old known sequence:

    if echo $SERVERS | grep -q $HOSTNAME ; then
        Server
    elif echo $CLIENTS | grep -q $HOSTNAME ; then
        Client
    else
        Fail
    fi

Ideas? Have a better name?

Comment 2 Fedora Admin XMLRPC Client 2014-09-02 12:13:17 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 3 Dalibor Pospíšil 2014-09-02 13:25:56 UTC
Is this RFE still valid?

Personally I don't like this approach to multihost. I would be happy if the multihost test could be written in expect-like way so more terminals open and commands executed serially step by step on server and client site in one block of code.

Example:

rlRun@server "service vsftpd start"
rlRun@client "lftp test@$SERVERS ..."

This is way harder solution but it is more understandable.

Comment 4 Dalibor Pospíšil 2015-03-24 15:57:31 UTC
Isn't it what distribution/sync does?

Comment 5 Dalibor Pospíšil 2015-06-12 17:02:20 UTC
I am for expect-like approach.

Comment 7 Ondrej Moriš 2015-12-01 12:59:50 UTC
(In reply to Petr Šplíchal from comment #0)

Hi, let me add a few notes here. 

First of all, I was quite disappointed with multihost testing capabilities in rhts packages. I had the following objections:

 * There should be no difference between running tests manually and 
   automatically by beaker, why should I modify runtest.sh and manually drive
   synchronization (by raising signals by pressing ENTER)? Manual testing is 
   used during test development and debugging and such need makes it really
   annoying.

 * Except synchronization signals there are no other mechanisms - eg. if I 
   want to exchange data between nodes, I have to create it from scratch using
   signals myself. 

We are using Multihost testing quite extensively for openswan/libreswan testing where most of the tests require two nodes (ipsec over loopback was dropped recently. Thus I wrote a simple beaker library /distribution/sync [1] (55 consumers in various components at the time being) to bypass aforementioned objections. It is completely independent on rhts synchronization and works for just two nodes (1 server, 1 client) which is sufficient for most multihost testing and the only multihost scenario supported by workflow-tomorrow.

I will now explain how it solves the original request.

> When writing a multihost test, quite a bit of the code is always
> the same:
> 
>     * find out if running in debugging mode or on RHTS
>     * optional manual setting of SERVERS & CLIENTS variables

You do not need such code with "sync" library. In the library there is no difference between manual execution and beaker automation. It always expect variables SERVERS and CLIENTS to be set, manual execution is simple:

# SERVERS=<server> CLIENTS=<client> make run (on both sides)

No DNS needed, FDQN or IP address can be used and hence you can easily use your virtual machines for test debugging, you only need to be in Red Hat network (to have access to /mnt/qa mount point, for more details, see [1]).

>     * deciding whether we are on a server or a client

Each node known its role, it is set during library initialization and it is clearly reported to the test log:

:: [ 05:20:11 ] :: sync: Setting syncSLEEP to 5 seconds
:: [ 05:20:11 ] :: sync: Setting syncTIMEOUT to 7200 seconds
:: [ 05:20:11 ] :: sync: Setting syncTEST to 584987
:: [ 05:20:11 ] :: sync: Setting syncTTL to 600 minutes
:: [ 05:20:11 ] :: sync: Setting syncCLIENT to "10.16.66.194"
:: [ 05:20:11 ] :: sync: Setting syncCLIENTv6 to "2620:52:0:1040:de:adff:febe:ef03"
:: [ 05:20:12 ] :: sync: Setting syncSERVER to "10.16.46.57"
:: [ 05:20:12 ] :: sync: Setting syncSERVERv6 to "2620:52:0:102f:5054:1ff:fe2c:e130"
:: [ 05:20:12 ] :: sync: Setting syncETH to "eth0"
:: [ 05:20:12 ] :: sync: Setting syncETHv6 to "eth0"
:: [ 05:20:12 ] :: sync: Setting syncROLE to "CLIENT"
:: [ 05:20:12 ] :: sync: Setting syncME to "10.16.66.194"
:: [ 05:20:12 ] :: sync: Setting syncMEv6 to "2620:52:0:1040:de:adff:febe:ef03"
:: [ 05:20:12 ] :: sync: Setting syncOTHER to "10.16.46.57"
:: [ 05:20:13 ] :: sync: Setting syncOTHERv6 to "2620:52:0:102f:5054:1ff:fe2c:e130"
:: [ 05:20:13 ] :: sync: Setting syncSHARE to /mnt/qa/scratch/omoris/sync
:: [ 05:20:13 ] :: sync: Shared synchronization storage cleared
:: [ 05:20:13 ] :: sync: CLIENT is ibm-z10-03.rhts.eng.bos.redhat.com (10.16.66.194)
:: [ 05:20:13 ] :: sync: SERVER is hp-dl380pgen8-02-vm-8.lab.bos.redhat.com (10.16.46.57)
:: [ 05:20:13 ] :: sync: This test runs as CLIENT
:: [ 05:20:13 ] :: sync: CLIENT clears all its data
:: [   PASS   ] :: Command 'rlImport distribution/sync' (Expected 0, got 0)

> I suggest to implement a rlMultihost (or rlRunMultihost?) function
> that would incorporate all these duplicating steps.
> 
> Suggested syntax: 
> 
>     rlMultihost Server Client [ServerHost ClientHost]
> 
>     * Server, Client - functions providing respective actions
>     * ServerHost, ClientHost - hostname to be set in devel mode

You can then use various methods to run test code on a desired node:

  a) syncIsClient && rlRun ... 

  b) if syncClient; then 
        rlPhaseStart "Client"
           ... 
        rlPhaseEnd
     fi

  c) syncRun "CLIENT" <params> "..." (equivalent of syncIsClient && rlRun <params> "...")
         
(or for server node analogously).

For more details, feel free to ping me, or see [1]. All in all, I do not think we need direct multihost support in beakerlib when we have "sync" beaker library which satisfies all the requirements.

Any comments?

[1] http://wiki.test.redhat.com/BaseOs/BeakerLibraries/distribution_sync

Comment 8 Petr Šplíchal 2016-01-04 14:03:59 UTC
Hi, I haven't been working on multihost tests for some time so
this is not priority for me any more. I can see that the library
covers most of the required functionality so feel free to close
this bug if it's fine to have a redhat-internal-only solution.


Note You need to log in before you can comment on or make changes to this bug.