Bug 2066899

Summary: After IP change on Tues Mar 22 Satellite manifest tasks fail with 'could not initialize proxy [org.candlepin.model.UpstreamConsumer#XXXXX] - no Session'
Product: Red Hat Satellite Reporter: Paul Dudley <pdudley>
Component: CandlepinAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED ERRATA QA Contact: Lai <ltran>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.10.4CC: ahumbe, cdonnell, ehelms, jalbrech, nmoumoul, redakkan, saydas
Target Milestone: 6.11.0Keywords: Reopened, Triaged
Target Release: Unused   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: candlepin-3.1.28-4, candlepin-4.0.17-1, candlepin-4.1.12-1, candlepin-4.2.1-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2069129 2069132 2069136 2069139 (view as bug list) Environment:
Last Closed: 2022-07-05 14:34:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2069129, 2069132, 2069136, 2069139    
Bug Blocks:    

Description Paul Dudley 2022-03-22 17:56:34 UTC 
After change noted at https://status.redhat.com/ for Mar 22 Satellite manifest refresh task fails with:
~~~
---
task:
  created: '2022-03-22T16:59:16+0000'
  updated: '2022-03-22T16:59:16+0000'
  id: 8a8d62617e73eca6017fb29188330793
  name: Import Manifest
  group: 
  origin: satellite.example.redhat.com
  executor: 
  principal: foreman_admin
  state: CREATED
  previousState: CREATED
  startTime: 
  endTime: 
  attempts: 0
  maxAttempts: 1
  statusPath: "/jobs/8a8d62617e73eca6017fb29188330793"
  resultData: 
  key: ImportJob
poll_attempts:
  total: 3
  failed: 3
Error:
 
Katello::Errors::CandlepinError
 
org.hibernate.LazyInitializationException: could not initialize proxy [org.candlepin.model.UpstreamConsumer#8a8d62617e62b62e017e62c006d00004] - no Session
 
---
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.1.42/app/lib/actions/candlepin/abstract_async_task.rb:42:in
  `check_for_errors!'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.1.42/app/lib/actions/candlepin/abstract_async_task.rb:36:in
  `poll_external_task'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action/polling.rb:100:in
  `poll_external_task_with_rescue'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action/polling.rb:22:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.1.42/app/lib/actions/candlepin/abstract_async_task.rb:9:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action.rb:572:in
  `block (3 levels) in execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:27:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:19:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.1.42/app/lib/actions/middleware/keep_session_id.rb:11:in
  `block in run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.1.42/app/lib/actions/middleware/keep_session_id.rb:23:in
  `with_session_id'"
...
~~~
Comment 3 Paul Dudley 2022-03-22 21:10:13 UTC 
Issue has been resolved by reverting a change done in the portal at subscription.rhsm.redhat.com - more info at https://status.redhat.com/ - specifically https://status.redhat.com/incidents/17jjzhtpn942

Anyone still encountering this issue is encouraged to reach out to RH support here: https://access.redhat.com/support/contact/technicalSupport

Marking bugzilla as closed.
Comment 4 Paul Dudley 2022-03-23 20:04:39 UTC 
Reopening BZ to ask that Satellite in the future be able to account for these issues, should they occur again. In such a case where subscription-manager can easily grab new certificates, Satellite should also be able to update it's manifest distributor certificates without intervention from the user.
Comment 6 Lai 2022-04-27 21:11:40 UTC 
steps to retest:

1. Import a manifest from stage
2. Delete manifest
3. Import manifest from prod environment

Expected results:
Import should be successful

Actual:
Import is successful

I've also checked the candlepin log and see that there's no errors.

Tested on 6.11_snap 18 with candlepin-4.1.12-1.el8sat.noarch on rhel8 and rhel7
Comment 10 errata-xmlrpc 2022-07-05 14:34:49 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5498