Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1982468

Summary: Unable to download latest insights-client Core Egg on clients after upgrading Satellite to 6.9.x
Product: Red Hat Satellite Reporter: Nikhil Gupta <ngupta>
Component: RH Cloud - InventoryAssignee: Shimon Shtein <sshtein>
Status: CLOSED DUPLICATE QA Contact: Jameer Pathan <jpathan>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.9.0CC: ahumbe, aruzicka, mmccune, peter.vreman, sshtein
Target Milestone: 6.10.0Keywords: Regression, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1983977 (view as bug list) Environment:
Last Closed: 2021-07-18 07:33:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Nikhil Gupta 2021-07-15 00:16:18 UTC
Description of problem:
After upgrading to 6.9.3, Insights is reporting that a large number of hosts connected to Satellite are using an out of date egg. Looking at the Satellite logs while an insights client is running, there is a 500 error retrieving an update egg file. The issue appears to be affecting the majority of systems connected to Satellite.

~~~
2021-07-15T03:50:50 [I|app|f6c23b6e] Started GET "/redhat_access/r/insights/platform/module-update-router/v1/channel?module=insights-core" for 1.2.3.4 at 2021-07-15 03:50:50 +1200
2021-07-15T03:50:50 [I|app|f6c23b6e] Processing by InsightsCloud::Api::MachineTelemetriesController#forward_request as JSON
2021-07-15T03:50:50 [I|app|f6c23b6e]   Parameters: {"module"=>"insights-core", "path"=>"platform/module-update-router/v1/channel"}
2021-07-15T03:50:50 [I|app|f6c23b6e] Completed 200 OK in 307ms (Views: 0.3ms | ActiveRecord: 3.9ms | Allocations: 4955)
2021-07-15T03:50:50 [I|app|33d2b72a] Started GET "/redhat_access/r/insights/v1/static/release/insights-core.egg" for 1.2.3.4 at 2021-07-15 03:50:50 +1200
2021-07-15T03:50:50 [I|app|33d2b72a] Processing by InsightsCloud::Api::MachineTelemetriesController#forward_request as 
2021-07-15T03:50:50 [I|app|33d2b72a]   Parameters: {"path"=>"v1/static/release/insights-core"}
2021-07-15T03:50:51 [W|app|33d2b72a] Action failed
2021-07-15T03:50:51 [I|app|33d2b72a] Backtrace for 'Action failed' error (RestClient::NotFound): 404 Not Found
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:223:in `exception_with_response'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:103:in `return!'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:809:in `process_result'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:725:in `block in transmit'
....
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/configuration.rb:228:in `call'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/server.rb:713:in `handle_request'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/server.rb:472:in `process_client'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/server.rb:328:in `block in run'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/thread_pool.rb:134:in `block in spawn_thread'
 33d2b72a | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2021-07-15T03:50:51 [I|app|33d2b72a]   Rendering api/v2/errors/standard_error.json.rabl within api/v2/layouts/error_layout
2021-07-15T03:50:51 [I|app|33d2b72a]   Rendered api/v2/errors/standard_error.json.rabl within api/v2/layouts/error_layout (Duration: 11.2ms | Allocations: 6536)
2021-07-15T03:50:51 [I|app|33d2b72a] Completed 500 Internal Server Error in 326ms (Views: 17.6ms | ActiveRecord: 1.0ms | Allocations: 17654)
~~~

Insights client output when checking for a new Egg gives a IE 500 result because the Satellite fails to proxy the request to cloud.redhat.com

Version-Release number of selected component (if applicable):
Red Hat Satellite 6.9.3

How reproducible:
Always

Steps to Reproduce:
1. Register the client with Insights


Actual results:
Insights core egg file is not updated on the clients.

Expected results:
Insights core egg should be updated automatically.

Comment 4 Peter Vreman 2021-07-15 07:50:03 UTC
Additional info i already provided in the Case

I checked the output again and noticed a different in the URLs:

The module router used  https://cert.cloud.redhat.com/api/module-update-router/v1/channel

I tried then from the browser/curl to use 'cert.cloud.redhat.com' instead of 'cloud.redhat.com' and then it works!

Broken URL: https://cloud.redhat.com/api/v1/static/release/insights-core.egg
Working URL: https://cert.cloud.redhat.com/api/v1/static/release/insights-core.egg


Note that it can also be the cloud.redhat.com not making the content available on the expected URLs

Comment 5 Peter Vreman 2021-07-15 10:33:53 UTC
It is also worth mentioning that foreman_rh_cloud 3.0.22 has introduced more features that have bugs, see https://bugzilla.redhat.com/show_bug.cgi?id=1979810

Comment 6 Mike McCune 2021-07-15 20:55:00 UTC
Engineering is engaged and will develop a fix for this ASAP, same for 1979810.

Comment 10 Shimon Shtein 2021-07-18 07:33:55 UTC

*** This bug has been marked as a duplicate of bug 1976754 ***

Comment 11 Red Hat Bugzilla 2023-09-18 00:28:24 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days