1246392 – foreman-proxy Failed to send SCAP results to the Foreman server: 500 "Internal Server Error"

Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1246392 - foreman-proxy Failed to send SCAP results to the Foreman server: 500 "Internal Server Error"

Summary: foreman-proxy Failed to send SCAP results to the Foreman server: 500 "Interna...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Satellite
Classification:	Red Hat
Component:	SCAP Plugin
Sub Component:
Version:	6.1.0
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	high
Target Milestone:	Unspecified
Assignee:	satellite6-bugs
QA Contact:	Kedar Bidarkar
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2015-07-24 08:17 UTC by Sebastian Hetze
Modified:	2019-12-16 04:50 UTC (History)
CC List:	8 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2016-07-27 09:16:51 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
Example of openscap report creating this error (228.12 KB, application/x-bzip) 2015-07-24 08:17 UTC, Sebastian Hetze	no flags	Details
Example of openscap report from same client working correctly (225.95 KB, application/x-bzip) 2015-07-24 08:20 UTC, Sebastian Hetze	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2016:1501	0	normal	SHIPPED_LIVE	Red Hat Satellite 6.2 Capsule and Server	2016-07-27 12:28:58 UTC

Description Sebastian Hetze 2015-07-24 08:17:55 UTC

Created attachment 1055647 [details]
Example of openscap report creating this error

Description of problem:

Ocassionally openscap reports are not digested properly by foreman-proxy.
Reports are uploaded correctly to /var/spool/foreman-proxy/openscap/arf but fail to be transferred to the foreman server.
This error occurs randomly in series of correctly handled reports from the same client. Once one bogus report occurs, foreman-proxy leaves all subsequent reports in the queue.

foreman-proxy debug output:

D, [2015-07-24T08:00:01.980290 #3822] DEBUG -- : Uploading 5c83c000e4119d92538ba04a8ba9c406c72916cfc5ce0d40aff5bc77eae0bdf5 to /api/v2/compliance/arf_reports/7be7b239-27bb-4b85-aa3d-7d58667ea5d2/1/2015-07-23
D, [2015-07-24T08:00:04.071839 #3822] DEBUG -- : {
  "error": {"message":"Could not read from bz_stream: BZ2_bzDecompress returns -4 [bz2.c:157]\nCould not create Result DataStream session: File is not Result DataStream. [ds_rds_session.c:54]\n"}
}
D, [2015-07-24T08:00:04.072150 #3822] DEBUG -- : /usr/share/ruby/net/http/response.rb:119:in `error!'
        /usr/share/ruby/net/http/response.rb:128:in `value'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:182:in `forward_arf_file'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:167:in `block in forward_date_dir'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:163:in `foreach'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:163:in `forward_date_dir'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:155:in `block in forward_policy_dir'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:152:in `foreach'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:152:in `forward_policy_dir'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:145:in `block in forward_cname_dir'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:142:in `foreach'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:142:in `forward_cname_dir'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:135:in `block in do'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:132:in `foreach'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:132:in `do'
        /usr/share/gems/gems/smart_proxy_openscap-0.3.0.9/lib/smart_proxy_openscap/openscap_lib.rb:79:in `send_spool_to_foreman'
        /usr/bin/smart-proxy-openscap-send:38:in `<main>'
E, [2015-07-24T08:00:04.072190 #3822] ERROR -- : Failed to send SCAP results to the Foreman server: 500 "Internal Server Error"

Version-Release number of selected component (if applicable):
foreman-proxy-1.7.2.5-1
ruby193-rubygem-foreman_openscap-0.3.2.10-1
puppet-foreman_scap_client-0.3.3-9

How reproducible:
Unfortunately this happens randomly.

Attached you find a report that generates the error. Copy it to /var/spool/foreman-proxy/openscap/arf/7be7b239-27bb-4b85-aa3d-7d58667ea5d2/1/2015-07-23/5c83c000e4119d92538ba04a8ba9c406c72916cfc5ce0d40aff5bc77eae0bdf5 and you should get the described behaviour. 

Actual results:
Listing of new reports in Satellite Hosts->Compliance->Reports stalls until you manually delete the bogus report from /var/spool/foreman-proxy/openscap/arf

Expected results:
- the actual reports do not look different and I expect all of them to work
- if somehow real bogus data is delivered to foreman-proxy, this one data set must be sorted out with meaningful message and processing of subsequent reports must proceed.

Additional info:

Comment 1 Sebastian Hetze 2015-07-24 08:20:10 UTC

Created attachment 1055648 [details]
Example of openscap report from same client working correctly

Comment 7 Kedar Bidarkar 2016-04-20 14:59:48 UTC

This issue being random is hard to reproduce reliably or to test.

But having faced this issue with sat6.1 few times myself  and me testing it with Sat6.2, I have not faced this issue with any sat6.2 builds.

Please re-open this bug if faced with sat6.2 beta or GA.

VERIFIED with sat62-snap8.1

Comment 8 Kedar Bidarkar 2016-04-20 15:08:44 UTC

I had raised a bug here for this issue, https://bugzilla.redhat.com/show_bug.cgi?id=1250065

Comment 10 errata-xmlrpc 2016-07-27 09:16:51 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501

Note You need to log in before you can comment on or make changes to this bug.