Bug 1370266

Summary: [RFE] Provide a way to reliably get number of failed asserts
Product: [Fedora] Fedora Reporter: Alois Mahdal <amahdal>
Component: beakerlibAssignee: Dalibor Pospíšil <dapospis>
Status: NEW --- QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: low    
Version: rawhideCC: azelinka, dapospis, mkyral, muller
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Alois Mahdal 2016-08-25 18:50:37 UTC
Description of problem
======================

I need to compare test state before/after a call to function that contains asserts, but the test (static file checker) may easily contain hundreds of other asserts earlier in the same phase.  Semantic of rlGetTestState and rlGetPhaseState is perfect but the design is unfortunate; they just won't work for values higher than 254.

beakerlib(1) (as of beakerlib-1.11-1.fc23) says:

> rlGetTestState
>
> Returns number of failed asserts in so far, 255 if there are more then 255 failures.
>
>     rlGetTestState
>
> rlGetPhaseState
>
> Returns number of failed asserts in current phase so far, 255 if there are more then 255
> failures.
>
>     rlGetPhaseState

(That's what you get when you abuse exit status.)


Proposal
========

I'm pretty sure those numbers are available;  there should be a supported way to get them.  Possibilities that come in mind:

 *  Tweak the above two to print the value to stdout instead of abusing
    exit status.

    Changing them directly could break something that relies on them being
    stdout-silent.  (But are they? I haven't checked.)  Introducing option
    could be easy althought better solution exist.

 *  Introduce their proper versions.

 *  Introduce a generic meta-info getter (`rlGet`):

     *  printing values to stdout,

     *  using ES properly, ie. to propagate error conditions,

     *  able to print also other useful info (can't think of anything
        as useful as fail number, but I'm sure something could be
        thrown in)

Comment 1 Dalibor Pospíšil 2016-08-29 13:42:21 UTC
At this moment I would prefer fixing this issue by passing the value using sone global variable similarly like rlRun passes rlRun_LOG if called with -s.

I'm afraid we won't implement any kind of ultimate rlGet function solving all wold's problems. But you are free to send a nice patch which we can base our discussions on.

Comment 2 Ales Zelinka 2016-08-30 09:25:51 UTC
I'd prefer reusing the existing functions. Maybe adding a param that would make the output more verbose, printing the exact numbers in a machine-friendly way.