Bug 1370266 - [RFE] Provide a way to reliably get number of failed asserts
Summary: [RFE] Provide a way to reliably get number of failed asserts
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: beakerlib
Version: rawhide
Hardware: Unspecified
OS: Unspecified
low
unspecified
Target Milestone: ---
Assignee: Dalibor Pospíšil
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-25 18:50 UTC by Alois Mahdal
Modified: 2018-01-29 23:55 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug


Attachments (Terms of Use)

Description Alois Mahdal 2016-08-25 18:50:37 UTC
Description of problem
======================

I need to compare test state before/after a call to function that contains asserts, but the test (static file checker) may easily contain hundreds of other asserts earlier in the same phase.  Semantic of rlGetTestState and rlGetPhaseState is perfect but the design is unfortunate; they just won't work for values higher than 254.

beakerlib(1) (as of beakerlib-1.11-1.fc23) says:

> rlGetTestState
>
> Returns number of failed asserts in so far, 255 if there are more then 255 failures.
>
>     rlGetTestState
>
> rlGetPhaseState
>
> Returns number of failed asserts in current phase so far, 255 if there are more then 255
> failures.
>
>     rlGetPhaseState

(That's what you get when you abuse exit status.)


Proposal
========

I'm pretty sure those numbers are available;  there should be a supported way to get them.  Possibilities that come in mind:

 *  Tweak the above two to print the value to stdout instead of abusing
    exit status.

    Changing them directly could break something that relies on them being
    stdout-silent.  (But are they? I haven't checked.)  Introducing option
    could be easy althought better solution exist.

 *  Introduce their proper versions.

 *  Introduce a generic meta-info getter (`rlGet`):

     *  printing values to stdout,

     *  using ES properly, ie. to propagate error conditions,

     *  able to print also other useful info (can't think of anything
        as useful as fail number, but I'm sure something could be
        thrown in)

Comment 1 Dalibor Pospíšil 2016-08-29 13:42:21 UTC
At this moment I would prefer fixing this issue by passing the value using sone global variable similarly like rlRun passes rlRun_LOG if called with -s.

I'm afraid we won't implement any kind of ultimate rlGet function solving all wold's problems. But you are free to send a nice patch which we can base our discussions on.

Comment 2 Ales Zelinka 2016-08-30 09:25:51 UTC
I'd prefer reusing the existing functions. Maybe adding a param that would make the output more verbose, printing the exact numbers in a machine-friendly way.


Note You need to log in before you can comment on or make changes to this bug.