Bug 1332702

Summary: smart-proxy-openscap-send with additional features - alert if file corrupt
Product: Red Hat Satellite Reporter: Waldirio M Pinheiro <wpinheir>
Component: SCAP PluginAssignee: Ondřej Pražák <oprazak>
Status: CLOSED ERRATA QA Contact: Jameer Pathan <jpathan>
Severity: high Docs Contact:
Priority: high    
Version: 6.1.8CC: bbuckingham, bkearney, cwelton, egolov, jcallaha, mhulan, mmccune, oprazak, sjagtap, vijsingh
Target Milestone: 6.8.0Keywords: Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: rubygem-smart_proxy_openscap-0.6.11-1 Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 12:57:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1542023, 1624072, 1861656    
Bug Blocks:    
Attachments:
Description Flags
Spool error Reports counter
none
Capsule show page with spool error
none
Verification_screenshot none

Description Waldirio M Pinheiro 2016-05-03 20:43:03 UTC
Description of problem:
After normal cron execution, the script smart-proxy-openscap-send in the Satellite server leave some files in the directory (/var/spool/foreman-proxy/openscap/arf), so the workaround should be rerun the command or check what file is corrupt.

Version-Release number of selected component (if applicable):
6.1.8

How reproducible:
100% (if you have any corrupt file in that structure)

Steps to Reproduce:
1. Configure openscap
2. Configure client, and force a new generation / send report
3. Change the report file in /var/spool/foreman-proxy/openscap/arf/* to became corrupt
4. On the server side, execute the script smart-proxy-openscap-send

Actual results:
Some files in the directory /var/spool/foreman-proxy/openscap/arf/* will not be processed and the admin will not be notified about it. So the report will still out of date and without any alert to the person who are responsible for the satellite environment.


Expected results:
Any kind of alert to the admin, should be in the Dashboard or Reports (inconsistent openscap reports for example).


Additional info:

Comment 1 Ondřej Pražák 2016-11-06 14:12:31 UTC
Created redmine issue http://projects.theforeman.org/issues/17240 from this bug

Comment 2 Satellite Program 2017-01-13 09:11:03 UTC
Upstream bug assigned to oprazak

Comment 3 Satellite Program 2017-02-21 11:11:30 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/17240 has been resolved.

Comment 11 Mike McCune 2018-03-09 18:09:22 UTC
ON_QA isn't an accurate status since this is blocked by the other bug mentioned here. Moving to ASSIGNED

Comment 12 Bryan Kearney 2018-06-21 17:13:46 UTC
Ths blocking bug is ON_QA, moving this to ON_QA as well.

Comment 13 Marek Hulan 2018-07-10 07:12:03 UTC
moving as mentioned in comment 12

Comment 14 Sanket Jagtap 2018-09-21 12:24:07 UTC
Build:Satellite 6.4.0 snap22

The corrupted file is not moved to a separate "corrupted" directory, Hence not detected by the proxy

Corrupting the report file
fallocate -l 2 -p -n arf/054ad748-d009-4213-b34c-c9c5c6d28822/1/1537532121/1c378cb68fb5255db5bf39f994c287135b350be2a0c371b0f6d3429834153677

Error on running smart-proxy-openscap-send command
smart-proxy-openscap-send 
/usr/share/gems/gems/openscap-0.4.7/lib/openscap/openscap.rb:34:in `raise!': Document is empty [oscap_source.c:301] (OpenSCAP::OpenSCAPError)
Entity: line 1: parser error : Document is empty
Unable to parse XML at: '/var/tmp/9609f49a-3d29-437f-8a96-271749c0b4d7-054ad748-d009-4213-b34c-c9c5c6d28822-1-1537532121-20180921-20142-1ybngeg' [oscap_source.c:303]
Could not create Result DataStream session: File is not Result DataStream. [ds_rds_session.c:54]
	from /usr/share/gems/gems/openscap-0.4.7/lib/openscap/ds/arf.rb:30:in `initialize'
	from /usr/share/gems/gems/smart_proxy_openscap-0.6.10/lib/smart_proxy_openscap/arf_json.rb:19:in `new'
	from /usr/share/gems/gems/smart_proxy_openscap-0.6.10/lib/smart_proxy_openscap/arf_json.rb:19:in `as_json'
	from /usr/bin/smart-proxy-arf-json:7:in `<main>'
Could not move file: No such file or directory - (/arf/054ad748-d009-4213-b34c-c9c5c6d28822/1/1537532121/1c378cb68fb5255db5bf39f994c287135b350be2a0c371b0f6d3429834153677, /var/lib/foreman-proxy/openscap/corrupted/arf/054ad748-d009-4213-b34c-c9c5c6d28822/1/1537532121/1c378cb68fb5255db5bf39f994c287135b350be2a0c371b0f6d3429834153677) See /usr/share/foreman-proxy/lib/../logs/openscap-send.log

Comment 16 Ondřej Pražák 2018-10-01 07:56:29 UTC
Connecting redmine issue http://projects.theforeman.org/issues/24508 from this bug

Comment 19 Sanket Jagtap 2018-10-03 11:43:11 UTC
Build: Satellite 6.4.0 snap25

I See the Proxy sent the corrupted report to corrupted Dir, but the UI error counter wasn't triggered.

The counter works on the capsule log that are present in UI. But the error log about the corupted report is not present on the UI. 

Moving it back it assigned as the UI counter is not updated with number of failed reports

Comment 21 Marek Hulan 2018-10-03 15:17:46 UTC
Sanket, if you mean the same counter as I do, it can't reflect this report and never did. Failed reports are reports that were successfully uploaded and contain failed rules. It means the actuals openscap check reports, host is failing to comply with the policy.

This BZ was addressing a problem, where the report was not uploaded at all, becauss of Satellite error. We don't have any internal error counters in Satellite as far as I know.

If I misunderstood, could you please upload a screenshot of page you have in mind? Thanks

Comment 22 Sanket Jagtap 2018-10-04 07:54:25 UTC
Created attachment 1490404 [details]
Spool error Reports counter

This is the field I was looking at.

Under capsules/<capsule-name>/Services tab

Comment 31 Ondřej Pražák 2019-11-19 14:34:11 UTC
Created attachment 1637708 [details]
Capsule show page with spool error

Ok, this should work in snaps for 6.7, corrupted report is moved to corrupted dir on capsule and server is able to detect it. Could QE verify?

Comment 35 Jameer Pathan 2020-09-03 14:01:55 UTC
Verified

Verified with:
- Satellite 6.8.0 snap 13

Test steps:
1. Have few scap reports from different hosts available in spool directory.
2. Corrupt report file in /var/spool/foreman-proxy/openscap/arf/* with 'fallocate -l 2 -p reportfile'
3. execute the script smart-proxy-openscap-send

Observation:
- "2 spool errors detected, inspect the appropriate file directly on Capsule" message on Satellite UI. 
(Go to Infrastructure > Capsule > select OpenSCAP capsule > Service )

- Corrupted reports are moved to /var/lib/foreman-proxy/openscap/corrupted/arf/

# smart-proxy-openscap-send
Traceback (most recent call last):
	4: from /usr/bin/smart-proxy-arf-json:7:in `<main>'
	3: from /opt/theforeman/tfm/root/usr/share/gems/gems/smart_proxy_openscap-0.7.3/lib/smart_proxy_openscap/arf_json.rb:19:in `as_json'
	2: from /opt/theforeman/tfm/root/usr/share/gems/gems/smart_proxy_openscap-0.7.3/lib/smart_proxy_openscap/arf_json.rb:19:in `new'
	1: from /opt/theforeman/tfm/root/usr/share/gems/gems/openscap-0.4.9/lib/openscap/ds/arf.rb:21:in `initialize'
/opt/theforeman/tfm/root/usr/share/gems/gems/openscap-0.4.9/lib/openscap/openscap.rb:25:in `raise!': Document is empty [oscap_source.c:302] (OpenSCAP::OpenSCAPError)
Entity: line 1: parser error : Document is empty
Unable to parse XML at: '/var/tmp/9b02828c-6b76-46b2-b44e-bf6ffa1a6609-ae1cfd9c-e9a9-4708-9ab0-f0b5a587f997-9-1599141116-20200903-11661-unraf9' [oscap_source.c:304]
Could not create Result DataStream session: File is not Result DataStream. [ds_rds_session.c:54]

# ls /var/lib/foreman-proxy/openscap/corrupted/arf/
7664e4d7-9573-4d2f-95e1-ec44bb6799f2  ae1cfd9c-e9a9-4708-9ab0-f0b5a587f997

Comment 36 Jameer Pathan 2020-09-03 14:13:29 UTC
Created attachment 1713627 [details]
Verification_screenshot

Comment 39 errata-xmlrpc 2020-10-27 12:57:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4366