Description of problem: Unable to log in via SSH to compute nodes with the heat-admin user Version-Release number of selected component (if applicable): OSP10 latest openstack-tripleo-heat-templates-5.2.0-13.el7ost.noarch puppet-tripleo-5.5.0-11.el7ost.noarch puppet-nova-9.5.0-4.el7ost.noarch How reproducible: 100% Steps to Reproduce: 1. Deploy OSP10 latest 2. SSH to one of the compute nodes [stack@undercloud-0 ~]$ ssh heat-admin.0.18 Connection closed by 192.168.0.18 Actual results: [stack@undercloud-0 ~]$ ssh heat-admin.0.18 -v OpenSSH_6.6.1, OpenSSL 1.0.1e-fips 11 Feb 2013 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 56: Applying options for * debug1: Connecting to 192.168.0.18 [192.168.0.18] port 22. debug1: Connection established. debug1: identity file /home/stack/.ssh/id_rsa type 1 debug1: identity file /home/stack/.ssh/id_rsa-cert type -1 debug1: identity file /home/stack/.ssh/id_dsa type -1 debug1: identity file /home/stack/.ssh/id_dsa-cert type -1 debug1: identity file /home/stack/.ssh/id_ecdsa type -1 debug1: identity file /home/stack/.ssh/id_ecdsa-cert type -1 debug1: identity file /home/stack/.ssh/id_ed25519 type -1 debug1: identity file /home/stack/.ssh/id_ed25519-cert type -1 debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_6.6.1 debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1 debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-ctr hmac-md5-etm none debug1: kex: client->server aes128-ctr hmac-md5-etm none debug1: kex: curve25519-sha256 need=16 dh_need=16 debug1: kex: curve25519-sha256 need=16 dh_need=16 debug1: sending SSH2_MSG_KEX_ECDH_INIT debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug1: Server host key: ECDSA ca:7f:c4:cb:b9:ea:de:2b:ea:e6:11:9a:d3:9b:d1:10 debug1: Host '192.168.0.18' is known and matches the ECDSA host key. debug1: Found key in /home/stack/.ssh/known_hosts:2 debug1: ssh_ecdsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received Connection closed by 192.168.0.18 [stack@undercloud-0 ~]$ Expected results: SSH connection is successfuly established. Additional info:
It seems that a typo was introduced in sshd_config on overcloud compute nodes while trying to configure ssh for live migration. It's not possible to ssh into compute nodes and sshd on computes complains about the bad line 21: "Match LocalAddress 192.168.24.16, User nova_migration" I believe that the comma is not supposed to be there and once I remove the comma, ssh starts to work as expected. There is possibly one more time a problem with comma at line: Match LocalAddress !192.168.24.16,! For sure sshd complains about the line 21 of the sshd_config where the comma should not be. If we have there more local migration addresses then they are supposed to be separated by comma but It's not the case of our setups. Example of created sshd_config by puppet: # File is managed by Puppet AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE AcceptEnv XMODIFIERS AuthorizedKeysFile .ssh/authorized_keys ChallengeResponseAuthentication no GSSAPIAuthentication yes GSSAPICleanupCredentials no HostKey /etc/ssh/ssh_host_rsa_key HostKey /etc/ssh/ssh_host_ecdsa_key HostKey /etc/ssh/ssh_host_ed25519_key PasswordAuthentication no PrintMotd no Subsystem sftp /usr/libexec/openssh/sftp-server SyslogFacility AUTHPRIV UsePAM yes UsePrivilegeSeparation sandbox X11Forwarding yes Match LocalAddress 192.168.24.16, User nova_migration AllowTcpForwarding no AuthorizedKeysFile /etc/nova/migration/authorized_keys ForceCommand /bin/nova-migration-wrapper PasswordAuthentication no X11Forwarding no Match LocalAddress !192.168.24.16,! DenyUsers nova_migration
The related reviews seem to be: https://review.openstack.org/#/c/458082/ https://review.openstack.org/#/c/458077/ It seems that the problem is that live_migration_ssh_inbound_addr is empty and thus tripleo::profile::base::nova::migration_ssh_localaddrs is something like: ['192.168.24.16', ''] From my /etc/puppet/hieradata/service_configs.yaml: . . . cold_migration_ssh_inbound_addr: 192.168.24.16 kernel_modules: { "nf_conntrack": {} } live_migration_ssh_inbound_addr: . . . tripleo::profile::base::nova::migration_ssh_localaddrs: [ "%{hiera('cold_migration_ssh_inbound_addr')}", "%{hiera('live_migration_ssh_inbound_addr')}" ] . .
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:1242