Description of problem: In terms of Test Results Evaluation Improvement Project [1] I'd like to be able to get additional informations from beaker test along with usual test result (PASS/FAIL). I don't want to change the code of existing tests, so this should be a beakerlib core functionality. Additional informations I'm interested in are NVRs of certain packages against which the test was run. How should it work? Let's have some special variable PKGNVR dedicated for this purpose. Example: Imagine we pass a parameter like this to the test: PKGNVR=nc;bind-utils After the test finishes, we'll get nc-1.84-22.el6.x86_64, bind-utils-9.7.3-8.P3.el6.x86_64 along with test results in the resulting XML. How should it be implemented? I think these new data should be stored in the beakerlib journal. Rationale: I'd like to be able to (over)write data acquired using the "PKGNVR" mechanism with some rl* function later in the test - during its run. That's because some tests may use yum or rpm, so I need a way to force-write the NVR to the journal during the test run. The aforementioned rl* function could be already existing rlAssertRpm after small modification. --------------- [1] https://wiki.test.redhat.com/BaseOs/Daemons/TestResutsEvaluationImprovementProject
Note that I created a rlAssertRpm modification request: bz772180.
Yes, I already have infrastructure support for this in the (yet unfinished) journal rewrite. Coming to your Beaker box in early 2012.
(In reply to comment #0) > Let's have some special variable PKGNVR dedicated for this purpose. > > Example: Imagine we pass a parameter like this to the test: > PKGNVR=nc;bind-utils After the test finishes, we'll get > nc-1.84-22.el6.x86_64, bind-utils-9.7.3-8.P3.el6.x86_64 along with > test results in the resulting XML. What about reporting NVRs of all packages mentioned in Requires?
A separate bugzilla for this is already filed: 772622.
Petr, do you have any estimation when this could be complete? Can I help you with the implementation?
I hope I will have the new journal in production until the end of FY.
Created attachment 575756 [details] proposed patch This patch reuses XML element "pkgdetails" which is already present in the journal. It adds package details for comma separated list of packages passed to the test via PKGNVR environmental variable: Examples: export PKGNVR=kernel,selinux-policy make run export PKGNVR=kernel,somenonsense make run unset PKGNVR make run ... Note that this also works when multiple packages with identical names (e.g. kernel) are installed. I hope that this would work in beaker like this: bkr workflow-tcms --task=/test/name --taskparam=PKGNVR='kernel,selinux-policy' ...
I've put in a slightly pythonized patch without the ugly try: (...) except: pass construction. http://git.fedorahosted.org/git?p=beakerlib.git;a=commit;h=7782b2db85f1fc9662817cfd077d4392d4c56177
Yeah, that's a bit better :) Thanks!
beakerlib-1.6-1 is already in fedora