Bug 1910412 - [RFE] Extend the ssh_args inside /etc/foreman-proxy/ansible.cfg file to have ansible required default values in Satellite 6
Summary: [RFE] Extend the ssh_args inside /etc/foreman-proxy/ansible.cfg file to have ...
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Ansible
Version: 6.8.0
Hardware: All
OS: All
Target Milestone: 6.10.0
Assignee: Ondřej Ezr
QA Contact: Danny Synk
Depends On:
TreeView+ depends on / blocked
Reported: 2020-12-23 19:10 UTC by Sayan Das
Modified: 2022-01-05 13:24 UTC (History)
4 users (show)

Fixed In Version: foreman-2.3.2
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2021-11-16 14:09:38 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 31553 0 Normal Closed Migrate ssh_args to ansible default for current installations 2021-02-09 15:30:24 UTC
Red Hat Product Errata RHSA-2021:4702 0 None None None 2021-11-16 14:09:56 UTC

Description Sayan Das 2020-12-23 19:10:44 UTC
1. Proposed title of this feature request

Extend the ssh_args inside /etc/foreman-proxy/ansible.cfg file to have ansible required default values in Satellite 6

2. What is the nature and description of the request?

Change in modification of ssh_args in /etc/foreman-proxy/ansible.cfg file for Satellite and Capsules by default.

Current value,
ssh_args = -o ProxyCommand=none

Expected value,
ssh_args = -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s

3. Why does the customer need this? (List the business requirements here)

By default when ansible initiates a connection for SSH, It uses "-C -o ControlMaster=auto -o ControlPersist=60s" . Example.
SSH: EXEC sshpass -d10 ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="root"' -o ConnectTimeout=30 -o ControlPath=/home/ansible/.ansible/cp/6caadfc83e client.example.com '/bin/sh -c '"'"'rm -f -r /tmp/ansible-root/ansible-tmp-1608743901.39-10561-116398582883036/ > /dev/null 2>&1 && sleep 0'"'"''

But the same thing does not happens with satellite as we have "-o ProxyCommand=none" hardcoded in ansible.cfg file.
SSH: EXEC ssh -vvv -o ProxyCommand=none -o StrictHostKeyChecking=no -o Port=22 -o 'IdentityFile="/usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 client.example.com '/bin/sh -c '"'"'echo ~root && sleep 0'"'"''

As per customer's testing, after making changes in ssh_args, We could see on the clients that the SSH socket was being reused which vastly increased performance (and decreased the run time of the playbooks). The playbook that was taking 16 mins to run now took about 4 - 8 mins. 

For my own testing, I saw that for the task which earlier ansible was taking 31 second, now it took around 21 seconds to finish. So this is a improvement that should be considered.

4. How would the customer like to achieve this? (List the functional requirements here)

We can easily achieve the same via following command :
satellite-installer --foreman-proxy-plugin-ansible-ssh-args "-o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s"

but the requirement here is to have that option by default present in /etc/foreman-proxy/ansible.cfg file. Since this is really an imporvement , many customer's might not be even aware of this fact and still working slower playbook\role execution.

Even in default ansible.cfg also we could see that ansible suggest using the same.
# cat /etc/ansible/ansible.cfg  | grep ssh_args
#ssh_args = -C -o ControlMaster=auto -o ControlPersist=60s

5. For each functional requirement listed, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented.

A) Deploy satellite or upgrade existing one.

B) Check the /etc/ansible/ansible.cfg  file for ssh_args parameter.

6. Is there already an existing RFE upstream or in Red Hat Bugzilla?

7. Does the customer have any specific timeline dependencies and which release would they like to target (i.e. RHEL5, RHEL6)?

As soon as possible

8. Is the sales team involved in this request and do they have any additional input?

9. List any affected packages or components.

- /etc/foreman-proxy/ansible.cfg
- tfm-rubygem-smart_proxy_ansible

10. Would the customer be able to assist in testing this functionality if implemented?

11. Additional information:

This RFE should be considered in favor of performance improvement of ansible based jobs unless we already had considered the same in past and had some complexities around the same.

Comment 2 Ondřej Ezr 2020-12-24 11:41:17 UTC
Created redmine issue https://projects.theforeman.org/issues/31552 from this bug

Comment 3 Ondřej Ezr 2020-12-24 12:07:21 UTC
Hmm after checking, this is the default for The fresh Satellite install.
But upgrade would not update to this value automatically. I've opened a ticket for that.

Comment 4 Sayan Das 2020-12-24 12:11:37 UTC
Hello Ondrez,

You are right.

* For new Satellite 6.8 installation the default value is the expected value.

* For Satellite 6.7 or older that is not applicable. They will still have the old value only.

* For satellite's being upgraded from 6.7 to 6.8, they will also still have the old value.

-- Sayan

Comment 5 Bryan Kearney 2020-12-24 21:42:44 UTC
Upstream bug assigned to oezr

Comment 6 Bryan Kearney 2020-12-24 21:42:46 UTC
Upstream bug assigned to oezr

Comment 7 Bryan Kearney 2021-01-13 13:29:41 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/31553 has been resolved.

Comment 8 Danny Synk 2021-07-06 17:21:14 UTC
Steps to Test:
1. Starting with a Satellite 6.7 instance, upgrade to Satellite 6.8 and 6.9 via the Red Hat CDN.
2. Register the Satellite 6.9 to internal Satellite compose server.
3. Configure repositories for upgrade to Satellite 6.10.
4. Prepare Satellite for Pulp 2 to Pulp 3 migration:

  # satellite-maintain prep-6.10-upgrade
  # satellite-maintain content prepare

5. Upgrade the Satellite to 6.10, snap 7.

Expected Results:
In /etc/foreman-proxy/ansible.cfg, the `ssh_args` setting has the value `-o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s`.

Actual Results:
In /etc/foreman-proxy/ansible.cfg, the `ssh_args` setting has the value `-o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s`:

  # grep ssh_args /etc/foreman-proxy/ansible.cfg
  ssh_args = -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s

Verified on Satellite 6.10, snap 7 (foreman-installer-2.5.1-1.el7sat.noarch).

Comment 9 Matthew LeSieur 2021-07-22 15:49:39 UTC
I discovered the missing "-o ControlMaster=auto -o ControlPersist=60s" ssh_args settings from /etc/foreman-proxy/ansible.cfg while troubleshooting a problem with our Palo Alto Networks (PAN) Firewall.  When SSH connection sharing is off, the PAN firewall was profiling the SSH connections from the Satellite server as a brute-force SSH attack.  PAN's description of the triggered threat is:

Threat ID: 40015
Application: SSH
Name: User Authentication Brute-force Attempt	
Description: If a session has the same source and destination but triggers our child signature, 31914, 20 times in 60 seconds, we call it is a brute force attack. The child signature, 31914 is alert on every connection on ssh server.

I was able to workaround the problem with the ssh_args setting from comment #8.  I am running Satellite 6.9.3.

Matthew LeSieur

Comment 12 errata-xmlrpc 2021-11-16 14:09:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.10 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.