Bug 1571784 - rstrnt-report-result is incompatible with beakerlib but set as command for reporting result
Summary: rstrnt-report-result is incompatible with beakerlib but set as command for re...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Restraint
Classification: Retired
Component: general
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: 0.1.35
Assignee: Matt Tyson 🤬
QA Contact: Jacob McKenzie
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-25 12:16 UTC by Pavel Holica
Modified: 2018-06-24 23:49 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-24 23:49:47 UTC


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Beaker Project Gerrit 6087 0 None MERGED make rstrnt-report-result accept rhts-report-result style arguments 2020-04-14 15:07:31 UTC

Description Pavel Holica 2018-04-25 12:16:48 UTC
Description of problem:

In file plugins/task_run.d/15_beakerlib there's:
export BEAKERLIB_COMMAND_REPORT_RESULT=/usr/bin/rstrnt-report-result

In beakerlib project in file /usr/share/beakerlib/testing.sh there's:
rlReport() {
    local testname="$1"
    local result="$(echo "$2" | tr '[:lower:]' '[:upper:]')"
    local score="$3"
    local logfile=${4:-$OUTPUTFILE}
...
    # report the result only if TESTID is set
    if [ -n "$TESTID" ] ; then
        $BEAKERLIB_COMMAND_REPORT_RESULT "$testname" "$result" "$logfile" "$score" \
            || rlLogError "rlReport: Failed to report the result"
    fi
}

But rstrnt-report-result accepts different arguments:
rstrnt-report-result [OPTION...] TASK_PATH RESULT SCORE
...
  -o, --outputfile=FILE
...

rhts-report-result from restraint-rhts solves this, but even if it's installed, it's not used by default.

Version-Release number of selected component (if applicable):
restraint-0.1.32-1
beakerlib-1.17-6

How reproducible:
always

Steps to Reproduce:
1. run test using beakerlib via restraint client
2. check index.html in the result directory

Actual results:
Instead of score, there's path to file, desired custom logfile is never uploaded.

Expected results:
Correct score is shown, custom logfile is uploaded.

Additional info:

Comment 1 Pavel Holica 2018-04-25 12:20:51 UTC
The problem is, that beakerlib does:
rstrnt-report-result "$testname" "$result" "$logfile" "$score"
but restraint expects:
rstrnt-report-result  TASK_PATH   RESULT    SCORE

Comment 2 Pavel Holica 2018-04-25 12:54:39 UTC
The workaround is to put following line to the test itself:
export BEAKERLIB_COMMAND_REPORT_RESULT=/usr/bin/rhts-report-result

Comment 3 Pavel Holica 2018-04-26 05:07:51 UTC
Sorry about not mentioning that the workaround requires to have restraint-rhts installed.

Comment 4 Matt Tyson 🤬 2018-04-30 06:35:34 UTC
This is actually preventing the beaker dogfood test suite from being run using restraint, but fixing it is a little more involved than I initially thought.

Comment 5 Matt Tyson 🤬 2018-05-04 04:46:50 UTC
I've modified rstrnt-report-result to have two argument parsing frontends.  One is the rstrnt-report-result style, the other is compatible with the old rhts-report-result style.

In the restraint RPM package there will be:

/usr/bin/rhts-report-result
/usr/bin/rstrnt-report-result

Under the hood they are the same binary.  It will detect the filename it was invoked as and process arguments accordingly.

The beakerlib plugin will call rhts-report-result as that is the argument format it is expecting.

Comment 7 Jacob McKenzie 2018-06-11 06:23:38 UTC
After running a job through restraint's client using beakerlib, this bug seems to be fixed. The 'index.html' has both a score number and custom log files.


Note You need to log in before you can comment on or make changes to this bug.