Bug 2094255

Summary: Configure Cloud Connector runs against an old hostname after a hostname change
Product: Red Hat Satellite Reporter: Lukáš Hellebrandt <lhellebr>
Component: satellite-change-hostnameAssignee: Evgeni Golov <egolov>
Status: CLOSED ERRATA QA Contact: Lukáš Hellebrandt <lhellebr>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.11.0CC: egolov, osousa, pcreech
Target Milestone: 6.11.0Keywords: Regression, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: katello-4.3.0-3 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-07-05 14:35:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Lukáš Hellebrandt 2022-06-07 09:49:19 UTC
Description of problem:
Cloud Connector installation job fails because it tries to run against an old (before hostname change + satellite-change-hostname) hostname instead of the correct one. The job is shown as executed against an incorrect host and this host is also used in the playbook itself:
```
  1:
[WARNING]: Callback disabled by environment. Disabling the Foreman callback
   2:
plugin.
   3:

   4:
PLAY [all] *********************************************************************
   5:

   6:
TASK [Gathering Facts] *********************************************************
   7:
fatal: [<OLD_FQDN>]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).", "unreachable": true}
   8:
PLAY RECAP *********************************************************************
   9:
<OLD_FQDN> : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0
  10:
Exit status: 1

```

Egolov found out that there are two hosts that are marked as "foreman=true" in the DB:
irb(main):003:0> a = ::Host.unscoped.search_for('infrastructure_facet.foreman = true')
=> #<ActiveRecord::Relation [#<Host::Managed id: 1, name: "<OLD_FQDN>", last_compile: nil, last_report: nil, updated_at: "2022-05-26 06:08:52", created_at: "2022-05-26 06:08:50", root_pass: nil, archite...
irb(main):004:0> a.length
=> 2

Also, egolov states that since rh_cloud 5.0.32, this is one of the ways RH Cloud detects the hostname. And indeed, with its previous version, this bug didn't manifest.

I am reporting the bug against satellite-change-hostname because the data is wrong in the database after running it. Yet, this is a regression (effectively, because it was possible to Configure Cloud Connector and now it is not) immediately caused by including a new version of rh_cloud in the snap.

Version-Release number of selected component (if applicable):
Sat 6.11 snap 22, tfm-rubygem-foreman_rh_cloud-5.0.37-1.el7sat.noarch
Regression

How reproducible:
Deterministic

Steps to Reproduce:
1. Have a Satellite prepared to get cloud connecor configured (e.g. ssh keys copied, s-m registered, manifest uploaded etc.)
2. Run satellite-change-hostname
3. Attempt Configure -> Inventory Upload -> Configure Cloud connector

Actual results:
Failure, job run against an old hostname

Expected results:
Job run against a new hostname

Additional info:
See also bug 2042480

Comment 1 Evgeni Golov 2022-06-07 10:01:54 UTC
Created redmine issue https://projects.theforeman.org/issues/35028 from this bug

Comment 2 Evgeni Golov 2022-06-07 13:02:49 UTC
the verification should be as easy as:

1. get a fresh 6.11 box (not from QE template!)
2. call satellite-change-hostname to rename it to a new name
3. open up `foreman-rake console` and check the following:
   1. ForemanRhCloud.foreman_host_name returns the right, new name
   2. ::Host.unscoped.search_for('infrastructure_facet.foreman = true').length is 1

Comment 3 Lukáš Hellebrandt 2022-06-14 09:19:14 UTC
Verified with Sat 6.11 snap 24.0.

# foreman-rake console
Loading production environment (Rails 6.0.4.7)
irb(main):001:0> ForemanRhCloud.foreman_host_name
=> "<NEW_FQDN>"
irb(main):002:0> ::Host.unscoped.search_for('infrastructure_facet.foreman = true').length
=> 1

And Configure -> Inventory Upload -> Configure Cloud Connector:

  58:
RUNNING HANDLER [redhat.satellite_operations.cloud_connector : Restart rhcd] ***
  59:
changed: [<NEW_FQDN>]

Comment 6 errata-xmlrpc 2022-07-05 14:35:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5498