Bug 1409460 - 'rhui-manager {cds,haproxy} add' wants {cds,haproxy} in known_hosts or ssh connection error
Summary: 'rhui-manager {cds,haproxy} add' wants {cds,haproxy} in known_hosts or ssh co...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Update Infrastructure for Cloud Providers
Classification: Red Hat
Component: Tools
Version: 3.0.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 3.0.x
Assignee: RHUI Bug List
QA Contact: Vratislav Hutsky
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-02 04:18 UTC by Irina Gulina
Modified: 2018-11-07 14:05 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-07 14:05:07 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:3520 0 None None None 2018-11-07 14:05:16 UTC

Description Irina Gulina 2017-01-02 04:18:33 UTC
Description of problem:
It's not possible to register cds and haproxy via 'rhui-manager cds/haproxy add'. It says: 
An SSH error occurred while connecting to ec2-user@{cds,hap}01.example.com:22:
Server '{cds,hap}01.example.com' not found in known_hosts.


Version-Release number of selected component (if applicable):
iso 20161220

How reproducible:
always 

Steps to Reproduce:
1. rhui {cds,haproxy} add {cds,hap}01.example.com ec2-user /root/.ssh/id_rsa_rhua

Actual results:
>> rhui cds add cds01.example.com ec2-user /root/.ssh/id_rsa_rhua

Checking that instance ports are reachable...
[localhost] local: yum install -y nc
[localhost] local: nc cds01.example.com 22 < /dev/null
Done.

An SSH error occurred while connecting to ec2-user.com:22:
Server 'cds01.example.com' not found in known_hosts.


>> rhui haproxy add hap01.example.com ec2-user /root/.ssh/id_rsa_rhua

Checking that instance ports are reachable...
[localhost] local: yum install -y nc
[localhost] local: nc hap01.example.com 22 < /dev/null
Done.

An SSH error occurred while connecting to ec2-user.com:22:
Server 'hap01.example.com' not found in known_hosts.




Meanwhile everything works fine in rhui-manager GNU:
---
rhui (loadbalancers) => a

Prior to registering a HAProxy Load-balancer, the instance must be provisioned with sshd running.

Hostname of the HAProxy Load-balancer instance to register:
hap01.example.com

Username with SSH access to hap01.example.com and sudo privileges:
ec2-user

Absolute path to an SSH private key to log into hap01.example.com as ec2-user:
/root/.ssh/id_rsa_rhua


Checking that instance ports are reachable...
[localhost] local: yum install -y nc
[localhost] local: nc hap01.example.com 22 < /dev/null
Done.


The SSH host key is not in the known_hosts file.

Please confirm that the following SSH host key fingerprint is correct for hap01.example.com:
  SSH host fingerprint (MD5): 2048 71:04:e2:3d:e7:ef:a8:0d:3d:96:bf:b7:e3:03:7c:78 hap01.example.com (ssh-rsa)
  SSH host fingerprint (SHA256): PDNvn/nmMAYnqNjkqxmtL6z8LFqkIM5U0i/TRDnUzgw (ssh-rsa)

Proceed? (y/n) y

The following HAProxy Load-balancer has been successfully added:

  Hostname:             hap01.example.com
  SSH Username:         ec2-user
  SSH Private Key:      /root/.ssh/id_rsa_rhua

The HAProxy Load-balancer will now be configured:

Checking that the RHUA services are reachable from the instance...
[ec2-user.com] sudo: yum install -d 0 -e 0 -y nc
[ec2-user.com] sudo: nc rhua.example.com 8140 < /dev/null
Done.


Installing Puppet on the HAProxy Load-balancer...
[ec2-user.com] sudo: yum install -y puppet
[ec2-user.com] sudo: puppet config set server rhua.example.com
Done.
[ec2-user.com] sudo: mkdir -p /etc/facter/facts.d
[ec2-user.com] put: /etc/puppet/rhui-secrets/rhui-custom-facts.json -> /etc/facter/facts.d/rhui-custom-facts.json

Setting up certificates...
[ec2-user.com] run: echo $HOME
[ec2-user.com] sudo: puppet config print certname
[ec2-user.com] sudo: rm -rf /var/lib/puppet/ssl
[ec2-user.com] sudo: mkdir -p /var/lib/puppet/ssl/{certs,private_keys}
[localhost] local: puppet cert clean hap01.example.com
[ec2-user.com] put: /var/lib/puppet/ssl/certs/ca.pem -> /home/ec2-user/ca.pem
[ec2-user.com] sudo: mv /home/ec2-user/ca.pem /var/lib/puppet/ssl/certs/ca.pem
[ec2-user.com] sudo: puppet agent --onetime --detailed-exitcodes --no-daemonize --no-usecacheonfailure
[ec2-user.com] run: echo $HOME
[ec2-user.com] sudo: cp -r /var/lib/puppet/ssl/certificate_requests/hap01.example.com.pem /home/ec2-user/hap01.example.com.pem
[ec2-user.com] sudo: chown -R $USER /home/ec2-user/hap01.example.com.pem
[ec2-user.com] download: /var/lib/puppet/ssl/ca/requests/hap01.example.com.pem <- /home/ec2-user/hap01.example.com.pem

Warning: Local file /var/lib/puppet/ssl/ca/requests/hap01.example.com.pem already exists and is being overwritten.

[ec2-user.com] sudo: rm -rf /home/ec2-user/hap01.example.com.pem
[localhost] local: puppet cert sign hap01.example.com
Done.

Installing and configuring the HAProxy Load-balancer...
[ec2-user.com] sudo: puppet agent --onetime --detailed-exitcodes --no-daemonize --no-usecacheonfailure
Done.

Ensuring that Apache and Crane are available...
[localhost] local: yum install -y nc
[localhost] local: nc hap01.example.com 443 < /dev/null
[localhost] local: nc hap01.example.com 5000 < /dev/null
Done.
The HAProxy Load-balancer was successfully configured.

------------------------------------------------------------------------------


Expected results:
cds and haproxy can be added via rhui CLI without being in known_hosts

Comment 3 Radek Bíba 2017-01-02 09:15:39 UTC
I believe this is the expected behavior in our setup. You must use the -u option. So:

rhui {cds,haproxy} add {cds,hap}01.example.com ec2-user /root/.ssh/id_rsa_rhua -u

Comment 4 Irina Gulina 2017-01-02 12:46:19 UTC
Then a message: 

An SSH error occurred while connecting to ec2-user@{cds,hap}01.example.com:22:
Server '{cds,hap}01.example.com' not found in known_hosts. Add it to known_hosts or use '-u/--unsafe' option. 

would be more helpful.

Comment 5 Irina Gulina 2017-01-02 13:02:43 UTC
or add a user dialog like in GUI: 

The SSH host key is not in the known_hosts file.

Please confirm that the following SSH host key fingerprint is correct for hap01.example.com:
  SSH host fingerprint (MD5): 2048 71:04:e2:3d:e7:ef:a8:0d:3d:96:bf:b7:e3:03:7c:78 hap01.example.com (ssh-rsa)
  SSH host fingerprint (SHA256): PDNvn/nmMAYnqNjkqxmtL6z8LFqkIM5U0i/TRDnUzgw (ssh-rsa)

Proceed? (y/n) y

Comment 6 Radek Bíba 2017-01-02 13:28:25 UTC
Adding a question to the CLI may break scripts that run unattended, in which case there's no one to press 'y'. I'd say the CLI should remain non-interactive by default.

Comment 8 Radek Bíba 2018-09-24 08:53:20 UTC
I've been looking into this recently and am quite confused. The rhui command refuses to add the node even if the host's key is already saved in known_hosts:

[root@rhua ~]# cat .ssh/known_hosts
hap01.example.com,10.103.216.138 ecdsa-sha2-nistp256 AAA<snip>kA=
[root@rhua ~]# rhui haproxy add hap01.example.com ec2-user /root/.ssh/id_rsa_rhua
 
Checking that instance ports are reachable...
[localhost] local: yum install -y nc
[localhost] local: nc hap01.example.com 22 < /dev/null
Done.
 
An SSH error occurred while connecting to ec2-user.com:22:
Server 'hap01.example.com' not found in known_hosts.

The SSH connection ought to work, though:

[root@rhua ~]# ssh -i /root/.ssh/id_rsa_rhua ec2-user.com
Last login: Mon Sep 24 07:34:57 2018 from ns01
[ec2-user@hap01 ~]$ 

So, something is fundamentally broken here.

Comment 9 Milan Kubík 2018-09-25 10:41:57 UTC
The paramiko client by default doesn't load the host keys. The command has the option to pass a hosts file for this purpose.
I can add the user's known_hosts so it would work similar to openssh client.

As for the error message when the fingerprint is not matched, the message comes from the paramiko library and there is no simple way to catch a specific error message and amend it.

Comment 19 errata-xmlrpc 2018-11-07 14:05:07 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3520


Note You need to log in before you can comment on or make changes to this bug.