Bug 1959932 - Insights not working after upgrade to satellite 6.9 (HTTP Status Code: 500)
Summary: Insights not working after upgrade to satellite 6.9 (HTTP Status Code: 500)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: RH Cloud - Insights
Version: 6.9.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: 6.9.3
Assignee: Shimon Shtein
QA Contact: Jameer Pathan
URL:
Whiteboard:
: 1969267 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-05-12 16:40 UTC by Amar Huchchanavar
Modified: 2023-04-21 15:39 UTC (History)
14 users (show)

Fixed In Version: tfm-rubygem-foreman_rh_cloud-3.0.22
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1972770 (view as bug list)
Environment:
Last Closed: 2021-07-01 14:56:52 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
tfm-rubygem-foreman_rh_cloud-3.0.22-1.el7sat.noarch.rpm (3.47 MB, application/x-rpm)
2021-06-18 15:23 UTC, Mike McCune
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github theforeman foreman_rh_cloud pull 547 0 None open Add proxy parameter to insights forwarded requests 2021-06-02 13:23:20 UTC
Red Hat Knowledge Base (Solution) 6028401 0 None None None 2021-05-12 20:25:08 UTC
Red Hat Product Errata RHBA-2021:2636 0 None None None 2021-07-01 14:57:22 UTC

Description Amar Huchchanavar 2021-05-12 16:40:25 UTC
Description of problem:
'Insights' clients are failing with below error:

~~~
# insights-client --test-connection
GET https://Satellite_URL:443/redhat_access/r/insights/v1/branch_info
GET branch_info status: 200
Running Connection Tests...
=== Begin Upload URL Connection Test ===
Testing: https://Satellite_URL:443/redhat_access/r/insights/uploads/
HTTP Status Code: 500
HTTP Status Text: Internal Server Error
HTTP Response Text: {
  "error": {"message":"Connection reset by peer - SSL_connect"}
}

Connection failed
=== End Upload URL Connection Test: FAILURE ===

=== Begin API URL Connection Test ===
Testing: https://Satellite_URL:443/redhat_access/r/insights/
HTTP Status Code: 200
HTTP Status Text: OK
HTTP Response Text: {}
Successfully connected to: https://Satellite_URL:443/redhat_access/r/insights/
=== End API URL Connection Test: SUCCESS ===

Connectivity tests completed with some errors
See /var/log/insights-client/insights-client.log for more details.
~~~

Error in /var/log/insights-client/insights-client.log

2021-05-06 13:56:59,730     INFO insights.client.connection === Begin Upload URL Connection Test ===
2021-05-06 13:56:59,730  NETWORK insights.client.connection Testing: https://Satellite_URL:443/redhat_access/r/insights/uploads/
2021-05-06 13:56:59,807  NETWORK insights.client.connection HTTP Status Code: 500
2021-05-06 13:56:59,807  NETWORK insights.client.connection HTTP Status Text: Internal Server Error
2021-05-06 13:56:59,807  NETWORK insights.client.connection HTTP Response Text: {
  "error": {"message":"Connection reset by peer - SSL_connect"}
}

2021-05-06 13:56:59,807     INFO insights.client.connection Connection failed



Version-Release number of selected component (if applicable):
Satellite 6.9

# rpm -q satellite 
satellite-6.9.1-1.el7sat.noarch


How reproducible:
Always
 ENV : Insights Client -> Satellite 6.9 -> Proxy -> cloud.redhat.com

Steps to Reproduce:
1. Install Satellite 6.9 
2. Install SQUID (on a different machine) OR use existing proxy setup
3. Configure Satellite 6.9 to use the Proxy.
   https://access.redhat.com/documentation/en-us/red_hat_satellite/6.9/html-single/installing_satellite_server_from_a_connected_network/index#configuring-satellite-with-an-http-proxy_satellite
4. Go to Satellite CLI.
   - disable DNS to reproduce the issue: 
     - Edit /etc/resolv.conf and comment out all entries so that satellite can not directly communicate outside.
      OR
     - block direct connections from satellite to WAN network.

5. Try #insights-client --test-con on satellite client 
   - It will fail with an error - HTTP Status Code: 500


Actual results:

~~~
#insights-client --test-connection

GET https://Satellite_URL:443/redhat_access/r/insights/v1/branch_info
GET branch_info status: 200
Running Connection Tests...
=== Begin Upload URL Connection Test ===
Testing: https://Satellite_URL:443/redhat_access/r/insights/uploads/
HTTP Status Code: 500
HTTP Status Text: Internal Server Error
HTTP Response Text: {
  "error": {"message":"Connection reset by peer - SSL_connect"}
}
~~~


Expected results:
The request should get routed through a proxy defined on the satellite server.



WORKAROUND:
- It works fine with global proxy settings defined in - Satellite Web UI > Administer > Settings > General > HTTP(S) proxy


Additional info:
- Satellite is unable to route the Insights request via proxy defined in Satellite UI -> Infrastructure -> HTTP proxies
- CDN sync appears to be working fine.

Comment 7 Hao Chang Yu 2021-06-08 08:51:36 UTC
*** Bug 1969267 has been marked as a duplicate of this bug. ***

Comment 8 Jameer Pathan 2021-06-17 14:44:58 UTC
Verified:

Verified with:
- Satellite 6.9.3 snap 3
- tfm-rubygem-foreman_rh_cloud-3.0.22-1.el7sat.noarch

Test steps:
- Create HTTP Proxy
- Go to Settings > Content and set "Default HTTP Proxy"
- Edit /etc/resolv.conf and comment out all entries so that satellite can not directly communicate outside.
- Register a host with satellite
- Try registering host with insights

Observations:
- insights-client register/unregister/test-connection/status requests are routed through 
http proxy set using "Default HTTP Proxy"(content) setting of Satellite.
- insights-client register/unregister/status requests are routed through 
http proxy set using "HTTP(S) proxy"(General) setting of Satellite.
- test-connection doesn't work for General "HTTP(S) proxy" setting. Filed BZ#1973285

Comment 9 Mike McCune 2021-06-18 15:22:48 UTC
*** Satellite 6.9.2 Hotfix Available ***

1) Download tfm-rubygem-foreman_rh_cloud-3.0.22-1.el7sat.noarch.rpm from this bugzilla to your Satellite


2) stop services:

satellite-maintain service stop

3) Install:

rpm -Uvh tfm-rubygem-foreman_rh_cloud-3.0.22-1.el7sat.noarch.rpm

4) restart:

satellite-maintain service start

5) Migrate database

foreman-rake db:migrate

6) resume operations

Comment 10 Mike McCune 2021-06-18 15:23:14 UTC
Created attachment 1792085 [details]
tfm-rubygem-foreman_rh_cloud-3.0.22-1.el7sat.noarch.rpm

Comment 16 errata-xmlrpc 2021-07-01 14:56:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Satellite 6.9.3 Async Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2636


Note You need to log in before you can comment on or make changes to this bug.