Description of problem: Currently the test skeleton created by beaker-wizard does not expect the task to be parametrized. For example to be able to run the same test against different versions of python: * system python (python) * parallel install (python3) * software collection (python27-python) The default template should be modified to allow asserting not only the fixed package name provided in the PACKAGE variable. This is now easy as BeakerLib has a fresh support for rlAssertRpm --all. I suggest to provide the following default template: # Include Beaker environment . /usr/bin/rhts-environment.sh || exit 1 . /usr/share/beakerlib/beakerlib.sh || exit 1 # Packages to be tested PACKAGES=${PACKAGES:-<run-for>} # Other required packages REQUIRES=${REQUIRES:-<additional-requires>} rlJournalStart rlPhaseStartSetup rlAssertRpm --all rlRun "TMP=\$(mktemp -d)" 0 "Creating tmp directory" rlRun "pushd $TMP" rlPhaseEnd rlPhaseStartTest rlRun "touch foo" 0 "Creating the foo test file" rlAssertExists "foo" rlRun "ls -l foo" 0 "Listing the foo test file" rlPhaseEnd rlPhaseStartCleanup rlRun "popd" rlRun "rm -r $TMP" 0 "Removing tmp directory" rlPhaseEnd rlJournalPrintText rlJournalEnd Where <run-for> will be substituted with the space-separated list of packages set in the RunFor and <additional-requires> with the rest of required packages (not duplicating those in RunFor). Let's also provide a new skeleton called 'parametrized' which will add other convenience features useful for testing components with non-fixed binaries. The new skeleton should look like this when 'python' is provided as the RunFor component: # Include Beaker environment . /usr/bin/rhts-environment.sh || exit 1 . /usr/share/beakerlib/beakerlib.sh || exit 1 # Packages to be tested PACKAGES=${PACKAGES:-python} # Other required packages REQUIRES=${REQUIRES:-} # Binary name parametrized PYTHON=${PYTHON:-python} rlJournalStart rlPhaseStartSetup rlAssertRpm --all rlAssertBinaryOrigin $PYTHON rlRun "TMP=\$(mktemp -d)" 0 "Creating tmp directory" rlRun "pushd $TMP" rlPhaseEnd rlPhaseStartTest rlRun -l "$PYTHON --version" 0 "Query binary version" rlPhaseEnd rlPhaseStartCleanup rlRun "popd" rlRun "rm -r $TMP" 0 "Removing tmp directory" rlPhaseEnd rlJournalPrintText rlJournalEnd Note that rlAssertBinaryOrigin is used to make sure that the binary under test is owned by one of rpms listed in PACKAGES. This is especially useful when testing software collections.
Dear Iveta, I've moved this item back to NEW since it is in ASSIGNED since 2013. Are you still working on it? Do you have problems? Can we help?
Hello Roman, I'm sorry. When I started to work on this there has been raised some questions regarding of dynamic test requires and this was huge topic for several months which led to unclear solution for this bug. I will bring this up for discussion and then decide next steps.
Update from today's Test Life Cycle meeting: Instead of modifying the default skeleton let's use the one below as a new skeleton called "parametrized" and leave the default skeleton unmodified. # Include Beaker environment . /usr/bin/rhts-environment.sh || exit 1 . /usr/share/beakerlib/beakerlib.sh || exit 1 # Packages to be tested PACKAGES=${PACKAGES:-<run-for>} # Other required packages REQUIRES=${REQUIRES:-<additional-requires>} rlJournalStart rlPhaseStartSetup rlAssertRpm --all rlRun "TMP=\$(mktemp -d)" 0 "Creating tmp directory" rlRun "pushd $TMP" rlPhaseEnd rlPhaseStartTest rlRun "touch foo" 0 "Creating the foo test file" rlAssertExists "foo" rlRun "ls -l foo" 0 "Listing the foo test file" rlPhaseEnd rlPhaseStartCleanup rlRun "popd" rlRun "rm -r $TMP" 0 "Removing tmp directory" rlPhaseEnd rlJournalPrintText rlJournalEnd
Created attachment 1112145 [details] proposed patch Proposed patch - adding parametrized skeleton and mentioning it in doc as available skeleton type
Patch has been moved to gerrit and is open for review: https://gerrit.beaker-project.org/#/c/4570/1
There are questions with this patch: * beaker-wizard currently does not accept multiple packages to be passed (e.g. -p argument). Therefore <runfor /> would always be a single package. Is that the desired behaviour? * There is currently no substitution for additional-requires. Is it perhaps a mistake and `requires` should be used or an additional substitution added? Updated patch can be found here: https://gerrit.beaker-project.org/#/c/4570/
Hello Roman, * yes, <runfor /> is one default package. The variable PACKAGES is filled in during job scheduling or if there is need to have there more packages, it's modified in a test. * hm, I left REQUIRES blank intentionally as this is mainly used in software collections and is also filled in during job scheduling. What does "additional-requires" contain? (or should contain?)
Dear Iveta, I thought it would contain additional required packages as outlined in comment 4 by Petr. If it's not needed I can remove it again?
Yes, I can see that. You know what, I'll discuss it with Petr to be really sure. Hopefully he will be recovered from illness soon.
Yes, both PACKAGES and REQUIRES should be pre-filled. Using <runfor/> and <requires/> seems to be the best option here. By the way, it is possible to provide several package for both required and runfor packages: $ cd /tmp $ beaker-wizard -yf -o a,b,c -q x,y,z /CoreOS/tmp/Sanity/a-few-descriptive-words ... Directory Sanity/a-few-descriptive-words created File Sanity/a-few-descriptive-words/PURPOSE written File Sanity/a-few-descriptive-words/runtest.sh written File Sanity/a-few-descriptive-words/Makefile written $ egrep 'RunFor|Requires' Sanity/a-few-descriptive-words/Makefile @echo "RunFor: a b c tmp" >> $(METADATA) @echo "Requires: tmp x y z" >> $(METADATA)
Created attachment 1115186 [details] proposed patch I've added <requires/> to fill in REQUIRES variable and adjusted runfor and requires variables so it's space separated in the skeleton.
(In reply to Petr Šplíchal from comment #11) > $ beaker-wizard -yf -o a,b,c -q x,y,z Okay, let's make sure the beaker-wizard test cases cover this functionality (passing multiple package names separated by commas) because I think none of us noticed that this exists. :-)
Dan, is there anything else needed from our side? Thanks.
(In reply to Iveta Wiedermann from comment #15) > Dan, is there anything else needed from our side? No nothing else is needed I think. Roman was working on this before he went on PTO and we were still just hammering out the right test cases on Gerrit before we merge the patch. I should have picked this up while he was gone, but I forgot, sorry about that.
This bug fix is included in beaker-client-22.2-0.git.19.07eb8fd which is currently available for download here: https://beaker-project.org/nightlies/release-22/
Note to verification: The main point of this bug was to introduce a new skeleton named "parametrized" with the template containing parametrization stuff. Comment #18 seems to verify only support for multiple packages provided in RunFor and Requires.
Yeah I guess a better verification would be: 1. Run: beaker-wizard -yf -o a,b,c -q x,y,z -s parametrized 2. Check Sanity/a-few-descriptive-words/runtest.sh # Packages to be tested PACKAGES=${PACKAGES:-a asdf b c} # Other required packages REQUIRES=${REQUIRES:-asdf x y z}
Beaker 22.2 has been released.