Bug 1387248

Summary:	oscap-docker incorrectly informs about incompliance of the assessed system
Product:	Red Hat Enterprise Linux 7	Reporter:	Matus Marhefka <mmarhefk>
Component:	openscap	Assignee:	Raphael Sanchez Prudencio <rsprudencio>
Status:	CLOSED ERRATA	QA Contact:	Matus Marhefka <mmarhefk>
Severity:	medium	Docs Contact:	Mirek Jahoda <mjahoda>
Priority:	high
Version:	7.3	CC:	mhaicman, mjahoda, mkolaja, mpreisle, openscap-maint, rsprudencio, swells, xdmoon
Target Milestone:	rc	Keywords:	ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:	In a case of a successful scan of an incompliant system, the oscap-docker command-line utility incorrectly reported an error. With this update, oscap-docker correctly assesses incompliant systems and prints which rules did not pass the scan to the evaluation results.	Story Points:	---
Clone Of:
Clones:	1410409 (view as bug list)		Environment:
Last Closed:	2017-08-01 08:45:48 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1410409

Description Matus Marhefka 2016-10-20 12:46:43 UTC

Description of problem:
Compliance scan of a Docker image/container using oscap-docker reports incorrectly that there was an error even if scan was successful but incompliance of the assessed system was found.

==========
$ oscap-docker image rhel7 xccdf eval --profile standard /usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml
WARNING: Skipping http://www.redhat.com/security/data/oval/Red_Hat_Enterprise_Linux_7.xml file which is referenced from XCCDF content

Command: oscap xccdf eval --profile standard /usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml failed!

Error was:

Command '['oscap', 'xccdf', 'eval', '--profile', 'standard', '/usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml']' returned non-zero exit status 2
==========

It is an issue in oscap-docker script when processing exit status of oscap which is according to oscap(8) man page:
Normally, the exit status is 0 when operation finished successfully and 1 otherwise. In cases when oscap performs evaluation of the system it may return 2 indicating success of the operation but incompliance of the assessed system.

Also, in case of incompliance, oscap-docker should print the evaluation results to pinpoint which rules did not pass the scan.


Version-Release number of selected component (if applicable):
openscap-utils-1.2.10-2.el7
scap-security-guide-0.1.30-3.el7


How reproducible:
always


Steps to Reproduce:
1. Run:
$ oscap-docker image rhel7 xccdf eval --profile standard /usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml


Actual results:
oscap-docker reports incorrectly that there was an error even if scan was successful but incompliance of the assessed system was found.

Expected results:
oscap-docker reports that scan was successful but incompliance of the assessed system was found. It will also print the evaluation results to pinpoint which rules did not pass the scan.


Additional info:

Comment 1 Matus Marhefka 2016-10-20 12:48:46 UTC

Also reported upstream.

Comment 3 Jan Černý 2016-11-01 14:08:35 UTC

Fixed upstream in https://github.com/OpenSCAP/openscap/commit/5ffdcf51b500494ac235a6a0160c126fc6f2144c

Comment 4 Shawn Wells 2016-11-21 16:29:33 UTC

Since this is a bug, will RHEL 7.3 errata be released (vs waiting for 7.4)?

Comment 10 Matus Marhefka 2017-04-26 12:26:55 UTC

VERIFIED as fixed for openscap-utils-1.2.14-1.el7

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Test: Scan a docker image
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   PASS   ] :: Command 'oscap-docker image rhel7:7.2 xccdf eval --profile xccdf_test_profile_failing --report ./report.html --results-arf ./results_arf.xml --results ./results.xml /mnt/tests/CoreOS/openscap/Sanity/oscap-docker/simple-xccdf.xml > ./output.stdout 2> ./output.stderr' (Expected 0, got 0)
:: [   PASS   ] :: File ./results.xml should exist 
:: [   PASS   ] :: Command 'file -b './results.xml' | grep '^XML' --quiet' (Expected 0, got 0)
:: [   PASS   ] :: File ./results_arf.xml should exist 
:: [   PASS   ] :: Command 'file -b './results_arf.xml' | grep '^XML' --quiet' (Expected 0, got 0)
:: [   PASS   ] :: File ./report.html should exist 
:: [   PASS   ] :: Command 'grep -q '<!DOCTYPE[^>]*html[^>]*>' './report.html'' (Expected 0, got 0)
:: [   PASS   ] :: File './output.stderr' should not contain '.' 
:: [   PASS   ] :: File './output.stdout' should contain 'elpmaxe.www_rule_2' 
:: [   PASS   ] :: File './output.stdout' should contain '^Result.*fail$'

Comment 11 errata-xmlrpc 2017-08-01 08:45:48 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2291