Bug 1034786

Summary: [RHEVM][hosted engine] SSH authentication failed during install
Product: Red Hat Enterprise Virtualization Manager Reporter: Martin Pavlik <mpavlik>
Component: ovirt-hosted-engine-setupAssignee: Sandro Bonazzola <sbonazzo>
Status: CLOSED ERRATA QA Contact: movciari
Severity: high Docs Contact:
Priority: high    
Version: 3.3.0CC: alonbl, dfediuck, didi, gklein, iheim, jbelka, lyarwood, oschreib, pablo.iranzo, pstehlik, scohen
Target Milestone: ---Keywords: Triaged
Target Release: 3.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: integration
Fixed In Version: ovirt-hosted-engine-setup-1.0.0-0.10.1.rc.el6ev Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-21 16:55:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
logs none

Description Martin Pavlik 2013-11-26 14:04:41 UTC 
Created attachment 829279 [details]
logs

Description of problem:
On clean install of hosted engine after confirmation that engine is installed in VM.

[ ERROR ] Cannot automatically add the host to the Default cluster: Cannot add Host. SSH authentication failed, verify authentication parameters are correct (Username/Password, public-key etc.) You may refer to the engine.log file for further details. 

Version-Release number of selected component (if applicable):
Red Hat Enterprise Virtualization Manager Version: 3.3.0-0.36.beta1.el6ev

How reproducible:
100%

Steps to Reproduce:
1. install hosted engine on fresh host (http://www.ovirt.org/Hosted_Engine_Howto#Fresh_Install)

Actual results:
[ ERROR ] Cannot automatically add the host to the Default cluster: Cannot add Host. SSH authentication failed, verify authentication parameters are correct (Username/Password, public-key etc.) You may refer to the engine.log file for further details. 

Expected results:
working install

Additional info:

2013-11-26 14:49:46 DEBUG otopi.plugins.ovirt_hosted_engine_setup.engine.add_host add_host._closeup:371 Cannot add the host to the Default cluster
Traceback (most recent call last):
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/engine/add_host.py", line 365, in _closeup
    override_iptables=True,
  File "/usr/lib/python2.6/site-packages/ovirtsdk/infrastructure/brokers.py", line 7752, in add
    headers={"Expect":expect, "Correlation-Id":correlation_id}
  File "/usr/lib/python2.6/site-packages/ovirtsdk/infrastructure/proxy.py", line 82, in add
    return self.request('POST', url, body, headers)
  File "/usr/lib/python2.6/site-packages/ovirtsdk/infrastructure/proxy.py", line 112, in request
    persistent_auth=self._persistent_auth)
  File "/usr/lib/python2.6/site-packages/ovirtsdk/infrastructure/proxy.py", line 134, in __doRequest
    persistent_auth=persistent_auth
  File "/usr/lib/python2.6/site-packages/ovirtsdk/web/connection.py", line 133, in doRequest
    raise RequestError, response
RequestError: ^M
status: 409^M
reason: Conflict^M
detail: Cannot add Host. SSH authentication failed, verify authentication parameters are correct (Username/Password, public-key etc.) You may refer to the engine.log file for further details.
2013-11-26 14:49:46 ERROR otopi.plugins.ovirt_hosted_engine_setup.engine.add_host add_host._closeup:378 Cannot automatically add the host to the Default cluster:
Comment 1 Doron Fediuck 2013-11-26 14:26:02 UTC 
This seems to be a change in rhel 6.5 behavior.

The reason is that sshd is unable to access the root authorized_keys
which so far had 0600 permissions. It is missing a read permission
for others to make it work:

[26/11/2013 15:02:26] <doron> Before"
[26/11/2013 15:02:29] <doron> -rw-------. 1 root root 409 Nov 26 14:59 /root/.ssh/authorized_keys
[26/11/2013 15:02:31] <doron> After:
[26/11/2013 15:02:37] <doron> -rw-r--r--. 1 root root 409 Nov 26 14:59 /root/.ssh/authorized_keys
[26/11/2013 15:02:42] <doron> now:
[26/11/2013 15:02:59] <doron> root@hosted-doron ~]#  ssh  -i  /etc/pki/ovirt-engine/keys/engine_id_rsa root.com
[26/11/2013 15:02:59] <doron> Last login: Tue Nov 26 15:00:40 2013 from hosted-doron.redhat.com
[26/11/2013 15:02:59] <doron> [root@sla-xxx ~]
Comment 2 Jiri Belka 2013-11-26 15:12:54 UTC 
type=AVC msg=audit(1385478716.042:4900): avc:  denied  { read } for  pid=32586 comm="sshd" name="authorized_keys" dev=dm-0 ino=3801109 scontext=unconfined_u:sy
stem_r:sshd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:admin_home_t:s0 tclass=file
Comment 3 Jiri Belka 2013-11-26 15:14:42 UTC 
# restorecon -RFv /root
restorecon reset /root/.rnd context unconfined_u:object_r:admin_home_t:s0->system_u:object_r:admin_home_t:s0
restorecon reset /root/.lesshst context unconfined_u:object_r:admin_home_t:s0->system_u:object_r:admin_home_t:s0
restorecon reset /root/.Xauthority context unconfined_u:object_r:xauth_home_t:s0->system_u:object_r:xauth_home_t:s0
restorecon reset /root/.bash_history context unconfined_u:object_r:admin_home_t:s0->system_u:object_r:admin_home_t:s0
restorecon reset /root/answerfile context unconfined_u:object_r:admin_home_t:s0->system_u:object_r:admin_home_t:s0
restorecon reset /root/.ssh context unconfined_u:object_r:admin_home_t:s0->system_u:object_r:ssh_home_t:s0
restorecon reset /root/.ssh/authorized_keys context unconfined_u:object_r:admin_home_t:s0->system_u:object_r:ssh_home_t:s0
restorecon reset /root/.recently-used.xbel context unconfined_u:object_r:admin_home_t:s0->system_u:object_r:admin_home_t:s0

and it works now.
Comment 4 Sandro Bonazzola 2013-11-26 15:20:59 UTC 
(In reply to Jiri Belka from comment #3)
> # restorecon -RFv /root
[cut]
> restorecon reset /root/.ssh context
> unconfined_u:object_r:admin_home_t:s0->system_u:object_r:ssh_home_t:s0
> restorecon reset /root/.ssh/authorized_keys context
> unconfined_u:object_r:admin_home_t:s0->system_u:object_r:ssh_home_t:s0

thanks! so it's a selinux issue not an access mode issue.
We'll need to check also AIO plugin for ensuring it's not affected too.
Comment 7 Sandro Bonazzola 2013-12-02 13:43:16 UTC 
Patches merged on upstream master and 1.0 branches.
Comment 9 errata-xmlrpc 2014-01-21 16:55:49 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0083.html