Bug 1257402 - [RFE] Confusing tests in blacklist configuration FILE
[RFE] Confusing tests in blacklist configuration FILE
Status: NEW
Product: rpmgrill
Classification: Community
Component: Documentation (Show other bugs)
0.28
Unspecified Unspecified
unspecified Severity medium
: ---
: ---
Assigned To: rpmdiff-maint@redhat.com
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-26 22:54 EDT by Meiying Li
Modified: 2015-08-27 01:25 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Meiying Li 2015-08-26 22:54:07 EDT
Description of problem:
Rpmgrill manpage is confusing users to configure blacklist YAML file, users don't know what tests are, how to find them.

Version-Release number of selected component (if applicable):
rpmgrill-0.28-2.fc22.noarch

How reproducible:
always

Steps to Reproduce:
1.The simplest way to start, for a given package NVR, is:

    $ NVR=coreutils-8.15-10.fc17
    $ ./bin/rpmgrill-fetch-build  $NVR my-$NVR
    $ ./bin/rpmgrill my-$NVR
    $ vim my-$NVR/rpmgrill.yaml      (or json or xml, if you prefer)
2.Find some results from rpmgrill.yaml file and add them to blacklist.yaml file, firstly man rpmgrill to learn how to add plugins and tests
3.

Actual results:
users don't know what test1 or test2 is, how to config it with the right test?
Because from rpmgrill manpage,we can find this:
--blacklist-config=FILE
           Blacklisted tests will not be reported by rpmgrill. Blacklist configuration FILE is in YAML format:

           blacklist:
             Plugin1: test1, test2
             Plugin2: test3
             ...

           and tells rpmgrill that it should not report fails for specified tests within certain plugins.

Expected results:
Show an example, what are the tests, so we need provide a clear template, for example, some results in rpmgrill.yaml as below:
  - '045 ManPages         : completed (1s)':
      - arch: 'i686,x86_64'
        code: ManPageMissing
        diag: 'No man page for <tt>/etc/DIR_COLORS</tt>'
        subpackage: coreutils
      - arch: 'i686,x86_64'
        code: ManPageMissing
        diag: 'No man page for <tt>/etc/DIR_COLORS.256color</tt>'
        subpackage: coreutils

According to the above results we want to configure them in blacklist yaml file:
           blacklist:
             ManPages: ManPageMissing

Then run:
rpmgrill --blacklist=blacklist.yaml  my-$NVR

at last, rpmgrill ignore the tests results successfully

Additional info:
Comment 1 yashi 2015-08-27 01:25:52 EDT
proposal as below:
1. add one option --list-tests=[ALL|all|specified_plugin_name]
1.1) will report tests for all the plugins when enter ALL/all 
e.g.
ManPages:
      ManPageMissing:

SpecFileSanity:
      ChangelogWrongWeekday

bla bla bla

1.2) will report tests for specified plugin when enter one plugin name
e.g. 
ManPages:
      ManPageMissing:

2. in the man page, add one human readable example with the real plugin and tests.

make sense?

Note You need to log in before you can comment on or make changes to this bug.