Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1607853

Summary: Instead of symlink there are two REX ssh keypairs in two distinct locations
Product: Red Hat Satellite Reporter: Lukas Pramuk <lpramuk>
Component: InstallationAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED ERRATA QA Contact: Lukas Pramuk <lpramuk>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.4CC: ehelms, mhulan
Target Milestone: 6.4.0Keywords: Regression, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: satellite-installer-6.4.0.7-1.beta Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-16 19:18:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Lukas Pramuk 2018-07-24 12:23:41 UTC
Description of problem:
Instead of symlink /usr/share/foreman-proxy/.ssh -> /var/lib/foreman-proxy/ssh there are two REX ssh keypairs in two distinct locations

Version-Release number of selected component (if applicable):
@satellite-6.4.0-10.beta.el7sat.noarch
foreman-installer-1.18.0-1.el7sat.noarch
rubygem-smart_proxy_remote_execution_ssh-0.2.0-3.el7sat.noarch

How reproducible:
deterministic

Steps to Reproduce:
1. Install Satellite

2. Check whether ssh keypairs match (should match because of symlink)
# diff /var/lib/foreman-proxy/ssh/ /usr/share/foreman-proxy/.ssh
diff /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy
2,26c2,26
< MIIEpAIBAAKCAQEAw8mSOBiSS/qQkK6R0XJhNpvOiyZyUU8/NWfHB61J+qYzMX9p
...
---
> MIIEowIBAAKCAQEAsft972uC6bFzIl4hKlBoFaa7iIelEplmgLAhVlBbyk4vjlmX
...
diff /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy.pub /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy.pub
1c1
< ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD...
---
> ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC...
 
Actual results:
no symlink, two distinct ssh keypairs in two distinct locations

Expected results:
symlink, only one ssh keypair symlinked to other location

Comment 2 Lukas Pramuk 2018-07-24 12:29:12 UTC
The rpm rubygem-smart_proxy_remote_execution_ssh-0.2.0-3.el7sat.noarch contains /usr/share/foreman-proxy/.ssh as a symlink.

So the breakage has to be done by installer, and here we go:
----
[ INFO 2018-07-24T12:51:23 main]  /Stage[main]/Foreman_proxy::Plugin::Remote_execution::Ssh/File[/usr/share/foreman-proxy/.ssh]: Starting to evaluate the resource
[DEBUG 2018-07-24T12:51:23 main]  /Stage[main]/Foreman_proxy::Plugin::Remote_execution::Ssh/File[/usr/share/foreman-proxy/.ssh]: Removing existing link for replacement with directory
[ WARN 2018-07-24T12:51:23 main]  /Stage[main]/Foreman_proxy::Plugin::Remote_execution::Ssh/File[/usr/share/foreman-proxy/.ssh]/ensure: ensure changed 'link' to 'directory'
[DEBUG 2018-07-24T12:51:23 main]  /Stage[main]/Foreman_proxy::Plugin::Remote_execution::Ssh/File[/usr/share/foreman-proxy/.ssh]: The container Class[Foreman_proxy::Plugin::Remote_execution::Ssh] will propagate my refresh event
----

>>> Installer ensures back changed 'link' to 'directory', so removes symlink and generates new ssh keypair and you end up with two different keypairs

Comment 3 Marek Hulan 2018-07-24 14:23:20 UTC
this should be now ready for testing in linked MR

Comment 4 Lukas Pramuk 2018-07-25 09:16:14 UTC
The consequence is that after upgrade to 6.4, REX on all existing hosts stop working!!!

Since existing ssh keypair was moved out to /var/lib/foreman-proxy/ssh and instead the symlink to it a new keypair is generated in /usr/share/foreman-proxy/.ssh

Comment 7 Lukas Pramuk 2018-08-01 11:51:46 UTC
VERIFIED.

@satellite-6.4.0-10.beta.el7sat.noarch (Snap14)
tfm-rubygem-foreman_ansible-2.2.5-1.el7sat.noarch
tfm-rubygem-foreman_ansible_core-2.1.1-1.el7sat.noarch
rubygem-smart_proxy_ansible-2.0.2-3.el7sat.noarch
ansible-2.6.1-1.el7ae.noarch

by manual reproducer from comment #0:

2) Check whether keypairs match: 
# diff /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy /usr/share/foreman-proxy/.ssh

# ll /usr/share/foreman-proxy/.ssh 
lrwxrwxrwx. 1 root root 26 Jul 25 05:43 /usr/share/foreman-proxy/.ssh -> /var/lib/foreman-proxy/ssh

>>> ssh keypairs match since there is a symlinked directory

3) Check installer default for ssh identity dir:
# satellite-installer -h | grep identity-dir
    --foreman-proxy-plugin-remote-execution-ssh-ssh-identity-dir  Directory where SSH keys are stored (current: "/var/lib/foreman-proxy/ssh")

>>> ssh identity dir default is correctly migrated to the new default value

Comment 8 Bryan Kearney 2018-10-16 19:18:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2927