Bug 2173155 - REX jobs through jump hosts are failing as ProxyCommand=none is hard coded
Summary: REX jobs through jump hosts are failing as ProxyCommand=none is hard coded
Keywords:
Status: NEW
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Remote Execution
Version: 6.12.1
Hardware: All
OS: All
unspecified
medium
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: Satellite QE Team
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-02-24 10:38 UTC by Nagoor Shaik
Modified: 2023-08-11 19:09 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:
Embargoed:
nshaik: needinfo-


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker SAT-18399 0 None None None 2023-06-14 12:30:41 UTC

Description Nagoor Shaik 2023-02-24 10:38:13 UTC
Description of problem:
Remote Execution jobs are failing when triggered actions goes through the jump host/bastion server.

Satellite ----> Jump Host/Bastion Server ---> Client

Even after configuring correct ssh parameters under ~foreman-proxy/.ssh/config REX jobs fails because the ProxyCommand=none is hard coded as per https://github.com/theforeman/smart_proxy_remote_execution_ssh/pull/89

Version-Release number of selected component (if applicable):
Satellite 6.12+

How reproducible:
100%

Steps to Reproduce:
1. Install Satellite 6.12
2. Register a client behind a NATed network (Ensure the only way to reach the host from Satellite is through a Jump Host)
3. Configure Jump Host configuration as below 

 # vi ~foreman-proxy/.ssh/config
Host bastion.example.com
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null
  IdentityFile ~foreman-proxy/.ssh/id_rsa_foreman_proxy

Host * !bastion.example.com
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null
  IdentityFile ~foreman-proxy/.ssh/id_rsa_foreman_proxy
  ProxyJump root.com

4. Ensure foreman-proxy SSH pub keys are copied to both bastion and client
5. Execute a simple REX job which fails with below error

~~~
1: Error initializing command: RuntimeError - Failed to establish connection to remote host, exit code: 255
2: Exit status: EXCEPTION
~~~

Actual results:
REX job fails

Expected results:
REX job should succeed and ProxyCommand argument shouldn't be hard-coded 

Additional info:
IMO, we need to revert this change https://github.com/theforeman/smart_proxy_remote_execution_ssh/pull/89

and this bug https://bugzilla.redhat.com/show_bug.cgi?id=2108637 can easily be addressed by just opening a documentation to include a switch `--no-ssh` argument with the ipa-client-install command.

Comment 1 Adam Ruzicka 2023-02-28 10:27:03 UTC
If I enroll a machine with ipa-client-install and later rerun it as ipa-client-install --no-ssh, will it fix things or does the argument need to be passed the first time the machine is enrolled?

Comment 2 Nagoor Shaik 2023-03-01 08:47:46 UTC
Unfortunately no, its not possible. It has to be passed in for the first time. In case a user has not configured it with --no-ssh option, then they have to run the ipa-client-install --uninstall first before re-running the "ipa-client-install --no-ssh" again.

See https://bugzilla.redhat.com/show_bug.cgi?id=1544379#c7

Comment 3 Adam Ruzicka 2023-03-01 12:23:08 UTC
That doesn't sound like a good user experience. Would having ssh options configurable the same way as we do for ansible be acceptable?

Comment 4 Nagoor Shaik 2023-03-02 04:27:11 UTC
Yes, that looks good plan. But, I strongly feel that --no-ssh is better option for Satellite 6, because it has to be executed just on the Satellite/Capsules which doesn't involve too much effort for the end-user.

Even if the customer has executed ipa-client-install without this option, they can just uninstall and then use the --no-ssh option which wouldn't cause any major outage and also it is common practice in IDM world as its recommended in https://bugzilla.redhat.com/show_bug.cgi?id=1544379#c7

Comment 6 Ben 2023-06-06 07:52:20 UTC
Just for reference, I've been fixing this for my own <= 6.12.2 (and now 6.13.1) install by commenting out the line

ssh_options << "-o ProxyCommand=none"

in file 

/usr/share/gems/gems/smart_proxy_remote_execution_ssh-0.10.1/lib/smart_proxy_remote_execution_ssh/multiplexed_ssh_connection.rb 

(previously under 6.12.2 it was in 

/usr/share/gems/gems/smart_proxy_remote_execution_ssh-0.7.3/lib/smart_proxy_remote_execution_ssh/runners/script_runner.rb)

I'd love not to have to do so as it messes up "satellite-maintain upgrade check" runs (-:


Note You need to log in before you can comment on or make changes to this bug.