Bug 1581211

Summary: host_key_checking = True prevents job to run
Product: Red Hat Satellite Reporter: Lukas Pramuk <lpramuk>
Component: Ansible - Configuration ManagementAssignee: Daniel Lobato Garcia <dlobatog>
Status: CLOSED ERRATA QA Contact: Lukas Pramuk <lpramuk>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.4CC: bkearney, jhutar, mhulan, pcreech
Target Milestone: 6.4.0Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: foreman_ansible-2.2.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-16 19:12:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Lukas Pramuk 2018-05-22 11:26:14 UTC
Description of problem:
host_key_checking = False in /etc/foreman-proxy/ansible.cfg is overridden by global system config /etc/ansible/ansible.cfg setting (=True by default)

Version-Release number of selected component (if applicable):
Sat6.4 Snap4
tfm-rubygem-foreman_ansible-2.0.4-1.el7sat.noarch (pimped to 2.1.2 with upstream gem)
tfm-rubygem-foreman_ansible_core-2.0.2-1.el7sat.noarch
rubygem-smart_proxy_ansible-2.0.2-3.el7sat.noarch
ansible-2.5.2-1.el7ae.noarch

How reproducible:
deterministic

Steps to Reproduce:
1. Install Satellite, have a host registered

2. Check the (currently) default setting 
# grep host_key_checking /etc/foreman-proxy/ansible.cfg
host_key_checking = False

3. Run ansible Command job

Target: host1.example.com
Scroll to bottom
   1:
[ERROR]:
   2:
   3:
PLAY [all] *********************************************************************
   4:
   5:
TASK [Gathering Facts] *********************************************************
   6:
The authenticity of host 'host1.example.com (192.168.100.10)' can't be established.
   7:
ECDSA key fingerprint is SHA256:hGgnIn9TBtWLKq3zZe7cpKDAlYaiUiD9d6xy53eV+DM.
   8:
ECDSA key fingerprint is MD5:3e:6e:a1:81:06:6d:4e:d7:2a:0d:c3:d8:df:dc:15:46.
   9:
Are you sure you want to continue connecting (yes/no)?


Actual results:
rex job stuck at host key check although host_key_checking = False in /etc/foreman-proxy/ansible.cfg

Expected results:
host_key_checking = False in /etc/foreman-proxy/ansible.cfg is in effect ( and not overridden by global system setting)

Comment 2 Daniel Lobato Garcia 2018-06-07 15:50:19 UTC
Connecting redmine issue http://projects.theforeman.org/issues/23731 from this bug

Comment 3 Lukas Pramuk 2018-06-12 13:48:55 UTC
Actual results:
always checking host - rex job stuck at host key check

Expected results:
reasonable way to ignore checking only for the first time - rex job passes

Comment 4 Satellite Program 2018-06-14 16:01:58 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/23731 has been resolved.

Comment 6 Lukas Pramuk 2018-06-27 12:16:18 UTC
VERIFIED.

@satellite-6.4.0-9.beta.el7sat.noarch
tfm-rubygem-foreman_ansible-2.2.0-1.el7sat.noarch
tfm-rubygem-foreman_ansible_core-2.1.0-1.el7sat.noarch
rubygem-smart_proxy_ansible-2.0.2-3.el7sat.noarch
ansible-2.6.0-0.5.rc5.el7ae.noarch

0) Have a host registered to SAT and REX ssh key set up on the host
# curl -k https://$SAT:9090/ssh/pubkey >> ~/.ssh/authorized_keys

1) Turn hostkey verification on:
(since we have BZ about ignoring per user config, set it in global ansible conf)
# grep ^host_key /etc/ansible/ansible.cfg 
host_key_checking = True

2) Clear foreman-proxy known hosts (to be sure)
# echo '' > ~foreman-proxy/.ssh/known_hosts

3) Run "Ansible Command" REX job with command 'uname'
>>> 100% success

4) Check content of knowhn hosts
# cat ~foreman-proxy/.ssh/known_hosts 
<FQDN>,<IP> ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwl/drCLTzg0prgoPlCV7WTBg4xJCdXoiAw7uCDLZhtmkyzf7l0X0+1hiBpAY+fhDBBabH/hWuTljK80zVq4l6fCNqNf5o4C83w4K7LGGERz+XX8xIXF5O65jv2QUPIiOIK5FwkkXxyK6yS7RcecnCxbGl8JHorfOLGNkzinDsLnhWKNXg4C6nKJLaHK7G59WxMoj01vuCI59RJB9elk8e+eTUz7dnmVNmDF/traDHQ5o1R9q9pJGL+L/RN6Lim8K3vUDyG2E59Hw27D0GSBzZR8lbXvcMyrZKY3K0dhOdcff8hUZx0uf3Zany41rdFqPR3LMqgKre5Ie4Ze3wTn3Vw==
<FQDN> ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwl/drCLTzg0prgoPlCV7WTBg4xJCdXoiAw7uCDLZhtmkyzf7l0X0+1hiBpAY+fhDBBabH/hWuTljK80zVq4l6fCNqNf5o4C83w4K7LGGERz+XX8xIXF5O65jv2QUPIiOIK5FwkkXxyK6yS7RcecnCxbGl8JHorfOLGNkzinDsLnhWKNXg4C6nKJLaHK7G59WxMoj01vuCI59RJB9elk8e+eTUz7dnmVNmDF/traDHQ5o1R9q9pJGL+L/RN6Lim8K3vUDyG2E59Hw27D0GSBzZR8lbXvcMyrZKY3K0dhOdcff8hUZx0uf3Zany41rdFqPR3LMqgKre5Ie4Ze3wTn3Vw==

Comment 7 Lukas Pramuk 2018-06-27 12:35:10 UTC
5) @HOST: Regenerate SSH host keys:
# rm -rfv /etc/ssh/*key* && /sbin/service sshd restart
removed `/etc/ssh/ssh_host_dsa_key'
removed `/etc/ssh/ssh_host_dsa_key.pub'
removed `/etc/ssh/ssh_host_key'
removed `/etc/ssh/ssh_host_key.pub'
removed `/etc/ssh/ssh_host_rsa_key'
removed `/etc/ssh/ssh_host_rsa_key.pub'
Stopping sshd:                                             [  OK  ]
Generating SSH2 RSA host key:                              [  OK  ]
Generating SSH1 RSA host key:                              [  OK  ]
Generating SSH2 DSA host key:                              [  OK  ]
Starting sshd:                                             [  OK  ]

6) Rerun the job that succeeded in step 3)
>>> 100% fail

----
   1:
PLAY [all] *********************************************************************
   2:
TASK [Gathering Facts] *********************************************************
   3:
fatal: [<HOST_FQDN>]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\r\n@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @\r\n@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\r\nIT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!\r\nSomeone could be eavesdropping on you right now (man-in-the-middle attack)!\r\nIt is also possible that a host key has just been changed.\r\nThe fingerprint for the RSA key sent by the remote host is\nSHA256:Y459EjGU2E7ognLLHOrxOgBQACiMYLUlfYW3UHSK9CQ.\r\nPlease contact your system administrator.\r\nAdd correct host key in /usr/share/foreman-proxy/.ssh/known_hosts to get rid of this message.\r\nOffending RSA key in /usr/share/foreman-proxy/.ssh/known_hosts:3\r\nRSA host key for <HOST_FQDN> has changed and you have requested strict checking.\r\nHost key verification failed.\r\n", "unreachable": true}
   4:
to retry, use: --limit @/tmp/foreman-playbook-9d7b02f7-bdf9-4d0c-9fea-16ba16fb22da.retry
   5:
   6:
PLAY RECAP *********************************************************************
   7:
<HOST_FQDN> : ok=0 changed=0 unreachable=1 failed=0
   8:
Exit status: 4
---

>>> Though Ansible REX provide reasonable way to store SSH hostkey on 1st run, it still prevents mitm attacks (i.e to connect when ssh host key has changed)

Comment 8 Bryan Kearney 2018-10-16 19:12:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2927