Bug 2123352

Summary: Updating katello-ca package does not update certs in yggdrasild service for REX pull mode client
Product: Red Hat Satellite Reporter: Pavel Moravec <pmoravec>
Component: Remote ExecutionAssignee: Adam Ruzicka <aruzicka>
Status: CLOSED ERRATA QA Contact: Peter Ondrejka <pondrejk>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.12.0CC: aruzicka, pcreech, saydas
Target Milestone: 6.12.0Keywords: Triaged
Target Release: Unused   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: foreman-installer-3.3.0.8-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-11-16 13:35:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Pavel Moravec 2022-09-01 12:50:11 UTC
Description of problem:
When using REX pull mode, yggdrasild service on a client gets populated certificates from sub-man via katello-pull-transport-migrate script. So far so good.

But when we update server CA on Satellite (or Capsule), katello-ca-consumer RPM is bumped and RHSM certs are updated via /usr/bin/katello-rhsm-consumer script - but yggdrasild certificates are not updated. And the service fails to authenticate to MQTT broker (until KPTM package is reinstalled, that is a workaround).


Version-Release number of selected component (if applicable):
Sat 6.12 beta/testathlon


How reproducible:
100%


Steps to Reproduce:
1. Have Sat with REX pull mode and a client running yggdrasild service
2. Update server CA on Sat
3. Update katello-ca-latest package on the client
4. Check yggdrasild service (maybe restart it..?)


Actual results:
4. Fails with error:
Sep 01 14:28:42 pmoravec-rhel8 yggdrasild[169940]: cannot connect to broker: network Error : x509: certificate signed by unknown authority


Expected results:
4. restart succeeds as yggdrasild certificates are updated.


Additional info:

Comment 1 Adam Ruzicka 2022-09-01 14:23:49 UTC
Wait, aren't we trying to move away from katello-ca-consumer rpm or am I confusing things? On my clients registered through global registration I don't have this package at all

Comment 2 Pavel Moravec 2022-09-01 14:49:38 UTC
(In reply to Adam Ruzicka from comment #1)
> Wait, aren't we trying to move away from katello-ca-consumer rpm or am I
> confusing things? On my clients registered through global registration I
> don't have this package at all

I dont know :) But if so, we must clearly state it in docs. Since a customer migrating from katello-agent to REX pull would hit this, otherwise.

(where is global registration described to let me play with it?)

Comment 3 Sayan Das 2022-09-04 10:05:18 UTC
Hello Adam,

I will add one point here [ considering capsule itself as a client of satellite server and custom certs are in question ]:

* I updated certs on my satellite.

* Now I am creating new cert bundle for my capsule using capsule-certs-generate:

# capsule-certs-generate --foreman-proxy-fqdn "$CAPSULE" --certs-tar  "~/$CAPSULE-certs.tar" 


This command itself will generate the new katello-ca-consumer-latest.noarch.rpm and instruct us to install the same on the target capsule in it's output.

###################

Preparing installation Done                                              
  Success!

  To finish the installation, follow these steps:

  If you do not have the Capsule registered to the Satellite instance, then please do the following:

  1. yum -y localinstall http://<sat fqdn>/pub/katello-ca-consumer-latest.noarch.rpm
  2. subscription-manager register --org "RedHat"
###################


And since we are used to this process, we will do "yum -y localinstall http://<sat fqdn>/pub/katello-ca-consumer-latest.noarch.rpm" on the capsule as well rather than trying to re-register it.

The same goes for client systems as well as Our own Product doc suggests to do this exact step for every client system connected with satellite\capsule:

--> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/installing_satellite_server_in_a_connected_network_environment/index#deploying-a-custom-ssl-certificate-to-hosts_satellite 


So Pavel's request makes perfect sense. We need to restart yggdrasild afterward. If it is not possible to do it automatically, then our SSL Certs installation doc needs to be updated, reflecting this requirement.


Also, about your statement :
~~
Wait, aren't we trying to move away from katello-ca-consumer rpm or am I confusing things? On my clients registered through global registration, I don't have this package at all
~~

Very true but not yet applicable for capsule-certs-generate command as it suggests doing katello-ca-consumer rpm installation or update, which is the easiest way to get the CA updated for Capsule's rhsm.

but if we don't want to use that, Then something equally quick should be suggested by the tool to use and update the RHSM CA on target capsules. 

I have raised an RFE for the same i.e. https://bugzilla.redhat.com/show_bug.cgi?id=2123352

Comment 5 Sayan Das 2022-09-05 11:53:34 UTC
I put the wrong RFE link earlier. It's https://bugzilla.redhat.com/show_bug.cgi?id=2124052 . I apologize for the confusion.

Comment 6 Bryan Kearney 2022-09-14 12:04:44 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/35486 has been resolved.

Comment 8 Peter Ondrejka 2022-10-05 10:53:15 UTC
Verified on satellite 6.12 snap 13, re-registration now also restarts yggdrasild

Comment 12 errata-xmlrpc 2022-11-16 13:35:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.12 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:8506