Bug 1287295

Summary: [RFE] verbose mode explaining why tests failed
Product: Red Hat Enterprise Linux 6 Reporter: Karl Hastings <kasmith>
Component: openscapAssignee: Šimon Lukašík <slukasik>
Status: CLOSED DUPLICATE QA Contact: Marek Haicman <mhaicman>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.7CC: jcerny, kasmith, mhaicman, sauchter
Target Milestone: rcKeywords: FutureFeature
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: openscap-1.2.7-1.el6 Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-01-18 14:23:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Karl Hastings 2015-12-01 22:20:58 UTC
Description of problem:
Provide more verbose information from Openscap for why something failed or was excluded.

E.g. if I run a scan and get

Title   Ensure gpgcheck Enabled For All Yum Package Repositories
Rule    ensure_gpgcheck_never_disabled
Ident   CCE-26647-8
Result  fail

Title   Ensure Software Patches Installed
Rule    security_patches_up_to_date
Ident   CCE-27635-2
Result  notchecked

How do I know which yum config is invalid?  And why was CCE-27635-2 not checked?  (in this case it was skipped in the content profile).

Version-Release number of selected component (if applicable):
openscap-scanner = 1.0.10-3.el6

Additional info:

customer dug into the xccdf and pulled out that it does a patetern check for <ind:pattern operation="pattern match">^\s*gpgcheck\s*=\s*0\s*$</ind:pattern>  looking threough yum I can find: 
> grep gpgcheck=0 /etc/yum.repos.d/*
/etc/yum.repos.d/custom.repo:gpgcheck=0

But there should be a way for oscap to display this information.

Comment 2 Šimon Lukašík 2015-12-02 08:55:27 UTC
Karl, we are preparing the verbose mode in upstream. This will be definitely brought to rhel customer with the next rebase.

In the meantime, please advice the customer to run:

    oscap xccdf eval

with

    --report ./myreport.html --results-arf ./myfullresults.arf.xml

options. In the detailed information is present in both files. HTML report would be much more easier to comprehend.

Comment 4 Šimon Lukašík 2015-12-15 08:44:03 UTC
The verbose mode has been implemented in upstream. See http://www.jan-cerny.cz/2015/12/09/verbose-mode-in-openscap-1-2-7/

Comment 7 Karl Hastings 2016-01-08 16:11:42 UTC
Marek,

Yes, that is exactly the type of data they are looking for.

The customer has said: "We are evaluting giving these results to the business but currently this would not be fit for that purpose."

They really want to provide a report to their internal customer that explains (simply) *why* the test failed, not just *that* it failed.

As mentioned in comment2 the HTML report is probably what they will want to provide to the business unit, so if the "Extended details" are in the HTML report, that would suffice.  But previously it wasn't there:

~~~
Result for Package Signature Checking is Not Disabled For Any Repos

Result: fail

Rule ID: rule-1008

Time: 2015-12-01 22:11

To ensure that signature checking is not disabled for any repos, ensure that the following line DOES NOT appear in any repo configuration files in /etc/yum.repos.d or elsewhere:

gpgcheck=0

Security identifiers

CCE-14813-0
~~~

Better would be something like:

~~~
Result for Package Signature Checking is Not Disabled For Any Repos

Result: fail

Rule ID: rule-1008

Time: 2015-12-01 22:11

This test failed because the following repo files disable Package Signature Checking in at least one repo by setting 'gpgcheck = 0':

/etc/yum.repos.d/beaker-harness.repo
/etc/yum.repos.d/beaker-client.repo.disabled
/etc/yum.repos.d/rhel-source.repo
/etc/yum.repos.d/beakerlib.repo.disabled
/etc/yum.repos.d/beaker.repo
/etc/yum.repos.d/rhel.repo
/etc/yum.repos.d/qa-tools.repo
/etc/yum.repos.d/rhel-debuginfo.repo
/etc/yum.repos.d/beaker-tasks.repo

Security identifiers

CCE-14813-0
~~~

Comment 11 Šimon Lukašík 2016-01-18 14:23:37 UTC
After discussion with the team, we have decided to make this bug a dupe of bug 1140240.

Both bugs basically request more detailed information in the HTML report, that is delivered in the current update.

*** This bug has been marked as a duplicate of bug 1140240 ***