Bug 1982468

Summary: Unable to download latest insights-client Core Egg on clients after upgrading Satellite to 6.9.x
Product: Red Hat Satellite Reporter: Nikhil Gupta <ngupta>
Component: RH Cloud - InventoryAssignee: Shimon Shtein <sshtein>
Status: CLOSED DUPLICATE QA Contact: Jameer Pathan <jpathan>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.9.0CC: ahumbe, aruzicka, mmccune, peter.vreman, sshtein
Target Milestone: 6.10.0Keywords: Regression, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1983977 (view as bug list) Environment:
Last Closed: 2021-07-18 07:33:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Nikhil Gupta 2021-07-15 00:16:18 UTC
Description of problem:
After upgrading to 6.9.3, Insights is reporting that a large number of hosts connected to Satellite are using an out of date egg. Looking at the Satellite logs while an insights client is running, there is a 500 error retrieving an update egg file. The issue appears to be affecting the majority of systems connected to Satellite.

~~~
2021-07-15T03:50:50 [I|app|f6c23b6e] Started GET "/redhat_access/r/insights/platform/module-update-router/v1/channel?module=insights-core" for 1.2.3.4 at 2021-07-15 03:50:50 +1200
2021-07-15T03:50:50 [I|app|f6c23b6e] Processing by InsightsCloud::Api::MachineTelemetriesController#forward_request as JSON
2021-07-15T03:50:50 [I|app|f6c23b6e]   Parameters: {"module"=>"insights-core", "path"=>"platform/module-update-router/v1/channel"}
2021-07-15T03:50:50 [I|app|f6c23b6e] Completed 200 OK in 307ms (Views: 0.3ms | ActiveRecord: 3.9ms | Allocations: 4955)
2021-07-15T03:50:50 [I|app|33d2b72a] Started GET "/redhat_access/r/insights/v1/static/release/insights-core.egg" for 1.2.3.4 at 2021-07-15 03:50:50 +1200
2021-07-15T03:50:50 [I|app|33d2b72a] Processing by InsightsCloud::Api::MachineTelemetriesController#forward_request as 
2021-07-15T03:50:50 [I|app|33d2b72a]   Parameters: {"path"=>"v1/static/release/insights-core"}
2021-07-15T03:50:51 [W|app|33d2b72a] Action failed
2021-07-15T03:50:51 [I|app|33d2b72a] Backtrace for 'Action failed' error (RestClient::NotFound): 404 Not Found
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:223:in `exception_with_response'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:103:in `return!'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:809:in `process_result'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:725:in `block in transmit'
....
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/configuration.rb:228:in `call'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/server.rb:713:in `handle_request'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/server.rb:472:in `process_client'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/server.rb:328:in `block in run'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/thread_pool.rb:134:in `block in spawn_thread'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2021-07-15T03:50:51 [I|app|33d2b72a]   Rendering api/v2/errors/standard_error.json.rabl within api/v2/layouts/error_layout
2021-07-15T03:50:51 [I|app|33d2b72a]   Rendered api/v2/errors/standard_error.json.rabl within api/v2/layouts/error_layout (Duration: 11.2ms | Allocations: 6536)
2021-07-15T03:50:51 [I|app|33d2b72a] Completed 500 Internal Server Error in 326ms (Views: 17.6ms | ActiveRecord: 1.0ms | Allocations: 17654)
~~~

Insights client output when checking for a new Egg gives a IE 500 result because the Satellite fails to proxy the request to cloud.redhat.com

Version-Release number of selected component (if applicable):
Red Hat Satellite 6.9.3

How reproducible:
Always

Steps to Reproduce:
1. Register the client with Insights


Actual results:
Insights core egg file is not updated on the clients.

Expected results:
Insights core egg should be updated automatically.

Comment 4 Peter Vreman 2021-07-15 07:50:03 UTC
Additional info i already provided in the Case

I checked the output again and noticed a different in the URLs:

The module router used  https://cert.cloud.redhat.com/api/module-update-router/v1/channel

I tried then from the browser/curl to use 'cert.cloud.redhat.com' instead of 'cloud.redhat.com' and then it works!

Broken URL: https://cloud.redhat.com/api/v1/static/release/insights-core.egg
Working URL: https://cert.cloud.redhat.com/api/v1/static/release/insights-core.egg


Note that it can also be the cloud.redhat.com not making the content available on the expected URLs

Comment 5 Peter Vreman 2021-07-15 10:33:53 UTC
It is also worth mentioning that foreman_rh_cloud 3.0.22 has introduced more features that have bugs, see https://bugzilla.redhat.com/show_bug.cgi?id=1979810

Comment 6 Mike McCune 2021-07-15 20:55:00 UTC
Engineering is engaged and will develop a fix for this ASAP, same for 1979810.

Comment 10 Shimon Shtein 2021-07-18 07:33:55 UTC

*** This bug has been marked as a duplicate of bug 1976754 ***

Comment 11 Red Hat Bugzilla 2023-09-18 00:28:24 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days