Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2028733 - katello-change-hostname fails to perform changes, leaving the system in an unusable state
Summary: katello-change-hostname fails to perform changes, leaving the system in an un...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: satellite-change-hostname
Version: 6.11.0
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: 6.11.0
Assignee: Evgeni Golov
QA Contact: Peter Ondrejka
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-12-03 04:43 UTC by Devendra Singh
Modified: 2022-07-05 14:30 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-07-05 14:30:33 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 34098 0 Normal New katello-change-hostname fails to perform changes, leaving the system in an unusable state 2021-12-07 10:58:24 UTC
Red Hat Product Errata RHSA-2022:5498 0 None None None 2022-07-05 14:30:55 UTC

Description Devendra Singh 2021-12-03 04:43:52 UTC 
Description of problem: foreman-proxy service failed with error "Exiting: No such file or directory @ rb_sysopen - /etc/foreman-proxy/ssl_key.pem"


Version-Release number of selected component (if applicable):
7.0 Snap1.1

How reproducible:
always

Steps to Reproduce:
1. Install Satellite 7.0 Snap1.1
2. Check the satellite services status.
3. All the services are in running state except foreman-proxy and which is getting failed with error "Exiting: No such file or directory @ rb_sysopen - /etc/foreman-proxy/ssl_key.pem"


Actual results:
Foreman-proxy service failed to start.

Expected results:
All the satellite services should be up and running.

Additional info:
Comment 3 Evgeni Golov 2021-12-06 11:33:43 UTC 
Looking at the proxy log, it did start up fine at 2021-12-02T12:38:26 and 2021-12-02T13:13:13, but failed at 2021-12-02T23:26:41.

So whatever happened in between, would be interesting to know.
Comment 4 Evgeni Golov 2021-12-06 12:49:08 UTC 
Looking at more logs, there seem to be more certs related issues on that machine:

tomcat:
Dec  2 23:25:09 dhcp-2-109 server: SEVERE: Failed to load keystore type PKCS12 with path /etc/candlepin/certs/keystore due to /etc/candlepin/certs/keystore (No such file or directory)
Dec  2 23:25:09 dhcp-2-109 server: java.io.FileNotFoundException: /etc/candlepin/certs/keystore (No such file or directory)
Comment 5 Evgeni Golov 2021-12-07 10:50:05 UTC 
The problem is not the installer, but `katello-change-hostname`.

When trying to change the hostname of an existing system, the tool bails out:

[root@ci-vm-10-0-101-91 ~]# satellite-change-hostname newhost.example.com -u admin -p changeme

Checking hostname validity

Checking overall health of server

Checking credentials

***WARNING*** This script will modify your system.
You will need to re-register any satellite clients registered to this system after script completion.
Capsules will have to be re-registered and reinstalled. If you are using custom certificates,
you will have to run the satellite-installer again with custom certificate options after this script completes.
 Have you taken the necessary precautions (backups, snapshots, etc...)?
Proceed with changing your hostname? [y/n]
y
Precheck passed
updating hostname in /etc/hostname
setting hostname
checking if hostname was changed

Updating default Capsule
Updating installation media paths
stopping services
removing old cert rpms
No Match for argument: ci-vm-10-0-101-91.hosted.upshift.rdu2.redhat.com-apache*
No Match for argument: ci-vm-10-0-101-91.hosted.upshift.rdu2.redhat.com-foreman-client*
No Match for argument: ci-vm-10-0-101-91.hosted.upshift.rdu2.redhat.com-foreman-proxy*
No Match for argument: ci-vm-10-0-101-91.hosted.upshift.rdu2.redhat.com-foreman-proxy-client*
No Match for argument: ci-vm-10-0-101-91.hosted.upshift.rdu2.redhat.com-puppet-client*
No Match for argument: ci-vm-10-0-101-91.hosted.upshift.rdu2.redhat.com-qpid-broker*
No Match for argument: ci-vm-10-0-101-91.hosted.upshift.rdu2.redhat.com-qpid-client-cert*
No Match for argument: ci-vm-10-0-101-91.hosted.upshift.rdu2.redhat.com-qpid-router-client*
No Match for argument: ci-vm-10-0-101-91.hosted.upshift.rdu2.redhat.com-qpid-router-server*
No Match for argument: ci-vm-10-0-101-91.hosted.upshift.rdu2.redhat.com-tomcat*
deleting old certs
backed up /var/www/html/pub to /var/www/html/pub/ci-vm-10-0-101-91.hosted.upshift.rdu2.redhat.com-20211207054627.backup
updating hostname in /etc/hosts
updating hostname in foreman installer scenarios
updating hostname in hammer configuration
backing up last_scenario.yaml
removing last_scenario.yaml
re-running the installer
satellite-installer --scenario satellite -v --disable-system-checks --certs-regenerate=true --foreman-proxy-register-in-foreman true
/usr/share/katello/hostname-change.rb:159:in `next_steps_message': undefined local variable or method `proxy' for #<KatelloUtilities::HostnameChange:0x00000001b9c408> (NameError)
	from /usr/share/katello/hostname-change.rb:570:in `installer_failure_message'
	from /usr/share/katello/hostname-change.rb:541:in `run'
	from /usr/sbin/satellite-change-hostname:23:in `<main>'

And *THEN* the certs are indeed missing:

[root@ci-vm-10-0-101-91 ~]# systemctl restart foreman-proxy
Job for foreman-proxy.service failed because the control process exited with error code. See "systemctl status foreman-proxy.service" and "journalctl -xe" for details.
[root@ci-vm-10-0-101-91 ~]# systemctl status foreman-proxy
● foreman-proxy.service - Foreman Proxy
   Loaded: loaded (/usr/lib/systemd/system/foreman-proxy.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/foreman-proxy.service.d
           └─90-limits.conf
   Active: failed (Result: exit-code) since Tue 2021-12-07 05:48:12 EST; 5s ago
  Process: 4130 ExecStart=/usr/share/foreman-proxy/bin/smart-proxy --no-daemonize (code=exited, status=1/FAILURE)
 Main PID: 4130 (code=exited, status=1/FAILURE)

Dec 07 05:48:10 newhost.example.com systemd[1]: Starting Foreman Proxy...
Dec 07 05:48:12 newhost.example.com smart-proxy[4130]: /opt/theforeman/tfm/root/usr/share/gems/gems/sequel-5.42.0/lib/sequel/adapters/sqlite.rb:114: warning: rb_check_safe_obj will be removed in Ruby 3.0
Dec 07 05:48:12 newhost.example.com smart-proxy[4130]: Errors detected on startup, see log for details. Exiting: No such file or directory @ rb_sysopen - /etc/foreman-proxy/ssl_key.pem
Dec 07 05:48:12 newhost.example.com systemd[1]: foreman-proxy.service: main process exited, code=exited, status=1/FAILURE
Dec 07 05:48:12 newhost.example.com systemd[1]: Failed to start Foreman Proxy.
Dec 07 05:48:12 newhost.example.com systemd[1]: Unit foreman-proxy.service entered failed state.
Dec 07 05:48:12 newhost.example.com systemd[1]: foreman-proxy.service failed.
Comment 6 Evgeni Golov 2021-12-07 10:58:23 UTC 
Created redmine issue https://projects.theforeman.org/issues/34098 from this bug
Comment 7 Peter Ondrejka 2022-01-14 15:28:37 UTC 
Verified on Satellite 7.0 snap 4
Comment 11 errata-xmlrpc 2022-07-05 14:30:33 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5498
Note You need to log in before you can comment on or make changes to this bug.