Bug 2143980

Summary: The "Configure Cloud Connector" job will fail to setup cloud-connector if remote_execution_ssh_user is set to a non-root user
Product: Red Hat Satellite Reporter: Sayan Das <saydas>
Component: RH CloudAssignee: Shimon Shtein <sshtein>
Status: CLOSED ERRATA QA Contact: Jameer Pathan <jpathan>
Severity: high Docs Contact:
Priority: high    
Version: 6.11.4CC: ahumbe, aruzicka, ehelms, jpathan, shivagup
Target Milestone: 6.15.0Keywords: PrioBumpGSS, Triaged
Target Release: Unused   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-04-23 17:12:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sayan Das 2022-11-18 15:33:50 UTC
Description of problem:

Despite following all the steps from "Chapter 3. Configuring your Satellite infrastructure to communicate with Insights" of https://access.redhat.com/documentation/en-us/red_hat_insights/2022/html-single/using_cloud_connector_to_remediate_issues_across_your_red_hat_satellite_infrastructure/index#configuring-your-satellite-infrastructure-to-communicate-with-insights , 

The "Configure Cloud Connector" job will fail to setup cloud-connector if remote_execution_ssh_user is set to a non-root user.

It will only be successful in one-shot if the remote_execution_ssh_user is set to root but that is not widely accepted. 


Version-Release number of selected component (if applicable):

Any version of Satellite 6 
( Tested on 6.9 , 6.10 , 6.11, 6.12 )


How reproducible:

Always


Steps to Reproduce:

1. Install a fresh Satellite 6.11 or 6.12

2. Confirm that it is properly registered with Red Hat portal

3. Change the value of "remote_execution_ssh_user" to "rexuser"
   # hammer settings set --name remote_execution_ssh_user --value rexuser


4. Execute the steps from "3.1. Uploading your host inventory from Satellite to Insights for RHEL" , specifically the installer command.

   # satellite-installer  --foreman-proxy-plugin-remote-execution-ssh-install-key true

  And then do a manual "Inventory Upload" from Satellite UI by clicking on Restart button for the required organization 


5. Go to Configure -> Inventory Upload page and click on "Configure Cloud Connector" button


Actual results:


Step 4 is successful but it shares the foreman-proxy ssh key to only the root user of satellite ( which is expected as per the code in /usr/share/foreman-installer/modules/foreman_proxy/manifests/plugin/remote_execution/ssh.pp )

~~
2022-11-18 20:43:56 [DEBUG ] [configure] Exec[install_ssh_key](provider=posix): Executing check 'grep -f /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy.pub /root/.ssh/authorized_keys'
2022-11-18 20:43:56 [DEBUG ] [configure] Executing: 'grep -f /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy.pub /root/.ssh/authorized_keys'
2022-11-18 20:43:56 [DEBUG ] [configure] Exec[install_ssh_key](provider=posix): Executing 'cat /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy.pub >> /root/.ssh/authorized_keys'
2022-11-18 20:43:56 [DEBUG ] [configure] Executing: 'cat /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy.pub >> /root/.ssh/authorized_keys'
2022-11-18 20:43:56 [INFO  ] [configure] /Stage[main]/Foreman_proxy::Plugin::Remote_execution::Ssh/Exec[install_ssh_key]/returns: executed successfully
2022-11-18 20:43:56 [DEBUG ] [configure] /Stage[main]/Foreman_proxy::Plugin::Remote_execution::Ssh/Exec[install_ssh_key]: The container Class[Foreman_proxy::Plugin::Remote_execution::Ssh] will propagate my refresh event
2022-11-18 20:43:56 [DEBUG ] [configure] /Stage[main]/Foreman_proxy::Plugin::Remote_execution::Ssh/Exec[install_ssh_key]: Evaluated in 0.02 seconds
~~


At Step 5, the cloud connector job execution will immediately fail 

~~
 13:
TASK [Gathering Facts] *********************************************************
  14:
fatal: [satellite611.example.com]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added 'satellite611.example.com,X.X.X.X' (ECDSA) to the list of known hosts.\r\nrexuser.com: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).", "unreachable": true}
  15:
PLAY RECAP *********************************************************************
  16:
satellite611.example.com : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0
  17:
Exit status: 1
~~


Simply because 

A) rexuser does not exists

B) Even if we will create it on satellite, It will not have the foreman-proxy SSH public key copied 


So the end-user needs to end-up using the steps from https://access.redhat.com/solutions/5196351 


Expected results:

If this is an expected behavior then that should be clearly documented in https://access.redhat.com/documentation/en-us/red_hat_insights/2022/html-single/using_cloud_connector_to_remediate_issues_across_your_red_hat_satellite_infrastructure/index#proc-uploading-your-host-inventory-from-satellite-to-insights_remediating-satellite

Or, The command "--foreman-proxy-plugin-remote-execution-ssh-install-key" flag should copy the SSH keys to the remote_execution_ssh_user i.e. rexuser in this case.


Additional info:

NA

Comment 2 Sayan Das 2022-11-18 16:02:11 UTC
Another solution could be to open the Satellite's entry from the All Hosts page and then add the "remote_execution_ssh_user" with value "root" in that profile.

But this will only work if the customer\end-user allows root login over SSH and have no issues doing so. 

Otherwise, for those who need to always use a non-root user for the REX , They would need to follow https://access.redhat.com/solutions/5196351 or similar steps to setup the non-root user on satellite itself for REX.

Comment 8 Brad Buckingham 2023-10-30 11:29:29 UTC
Bulk setting Target Milestone = 6.15.0 where sat-6.15.0+ is set.

Comment 13 errata-xmlrpc 2024-04-23 17:12:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.15.0 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2024:2010