Bug 1427149 - [downstream clone - 4.0.7] Sshd.service could not work normally after upgrade
Summary: [downstream clone - 4.0.7] Sshd.service could not work normally after upgrade
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: imgbased
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ovirt-4.0.7
: ---
Assignee: Ryan Barry
QA Contact: jianwu
URL:
Whiteboard:
Depends On: 1426151
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-02-27 13:23 UTC by rhev-integ
Modified: 2017-04-17 12:19 UTC (History)
19 users (show)

Fixed In Version: imgbased-0.8.15-0.1.el7ev
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1426151
Environment:
Last Closed: 2017-03-16 15:40:08 UTC
oVirt Team: Node
Target Upstream Version:
rbarry: needinfo-


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:0549 normal SHIPPED_LIVE Moderate: redhat-virtualization-host security and bug fix update 2017-03-16 19:26:25 UTC
oVirt gerrit 73185 master MERGED utils: add checksum_only option to rsync 2017-03-02 17:28:09 UTC
oVirt gerrit 73186 master MERGED osupdater: always copy essential files 2017-03-02 17:28:21 UTC
oVirt gerrit 73187 ovirt-4.1 MERGED utils: add checksum_only option to rsync 2017-03-02 17:28:49 UTC
oVirt gerrit 73188 ovirt-4.0 MERGED utils: add checksum_only option to rsync 2017-03-02 17:26:31 UTC
oVirt gerrit 73189 ovirt-4.1 MERGED osupdater: always copy essential files 2017-03-02 17:28:54 UTC
oVirt gerrit 73190 ovirt-4.0 MERGED osupdater: always copy essential files 2017-03-02 17:26:46 UTC

Description rhev-integ 2017-02-27 13:23:30 UTC
+++ This bug is an upstream to downstream clone. The original bug is: +++
+++   bug 1426151 +++
======================================================================

Description of problem:
After upgrade, sshd.service cannot work which happens only from redhat-virtualization-host-4.0-20161116.1.x86_64 to latest RHVH 4.0

Version-Release number of selected component (if applicable):
Before upgrade:
redhat-virtualization-host-4.0-20161116.1.x86_64(el7.3)
After upgrade:
redhat-virtualization-host-4.0-20170222.0.x86_64
kernel-3.10.0-514.6.2.el7.x86_64
imgbased-0.8.13-0.1.el7ev.noarch

How reproducible:
100%
regression bug


Steps to Reproduce:
1. Install redhat-virtualization-host-4.0-20161116.1.x86_64(el7.3) via anaconda
2. Reboot and log into this system
3. Set local repo and run #yum update
4. Reboot into new build
5. Run #systemctl status sshd

Actual results:
After step 5, sshd.service get something wrong and cannot access this host via ssh way.
#systemctl status sshd
service sshd status
● sshd.service - OpenSSH server daemon
   Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2017-02-23 15:28:43 CST; 1h 45min ago
     Docs: man:sshd(8)
           man:sshd_config(5)
  Process: 1465 ExecStart=/usr/sbin/sshd $OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 1585 (sshd)
   CGroup: /system.slice/sshd.service
           └─1585 /usr/sbin/sshd

Feb 23 15:29:28 dhcp-10-229.nay.redhat.com sshd[2898]: error: Permissions 0640 for '/etc/ssh/ssh_host_ecdsa_key' are too open.
Feb 23 15:29:28 dhcp-10-229.nay.redhat.com sshd[2898]: error: It is required that your private key files are NOT accessible by others.
Feb 23 15:29:28 dhcp-10-229.nay.redhat.com sshd[2898]: error: This private key will be ignored.
Feb 23 15:29:28 dhcp-10-229.nay.redhat.com sshd[2898]: error: bad permissions: ignore key: /etc/ssh/ssh_host_ecdsa_key
Feb 23 15:29:28 dhcp-10-229.nay.redhat.com sshd[2898]: error: Could not load host key: /etc/ssh/ssh_host_ecdsa_key
Feb 23 15:29:28 dhcp-10-229.nay.redhat.com sshd[2898]: error: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Feb 23 15:29:28 dhcp-10-229.nay.redhat.com sshd[2898]: error: @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
Feb 23 15:29:28 dhcp-10-229.nay.redhat.com sshd[2898]: error: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Feb 23 15:29:28 dhcp-10-229.nay.redhat.com sshd[2898]: error: Permissions 0640 for '/etc/ssh/ssh_host_ed25519_key' are too open.
Feb 23 15:29:28 dhcp-10-229.nay.redhat.com sshd[2898]: error: It is required that your private key files are NOT accessible by others.

Expected results:
After step 5, sshd.service run normally and could access this host via ssh way

Additional info:
1. It does not get wrong about sshd.service when upgrade from redhat-virtualization-host-4.0-20160919.0.x86_64(el7.2) to the latest RHVH 4.0
2. libssh is missing in latest 4.0.7 build. Not sure it is related for this bug
   *-libssh-0.7.1-2.el7.x86_64*

(Originally by Jian Wu)

Comment 1 rhev-integ 2017-02-27 13:23:42 UTC
No such issue happen in redhat-virtualization-host-4.0-20170201.0, so I thinks it is a regression bug.

(Originally by Jian Wu)

Comment 3 rhev-integ 2017-02-27 13:23:49 UTC
Created attachment 1256865 [details]
log 1 to RHVH

(Originally by Jian Wu)

Comment 4 rhev-integ 2017-02-27 13:23:56 UTC
libssh was added (and removed) as a dependency of cockpit.

I can't reproduce this, and the permissions here are exactly the same as a clean install. Were any additional steps taken? Some problem with the environment?

The logs look like there's an attempt to directly authenticate using /etc/ssh/ssh_host_ecdsa_key as a priv key, which will not even work on a base install at 0640 (these are the default permissions on EL distros and fedora)

Can you please post exact steps to reproduce and the complete "journalctl -u sshd.service" log?

---------------------------------------------------------------------------------------

[root@localhost ~]# imgbase layout
rhvh-4.0-0.20161116.0
 +- rhvh-4.0-0.20161116.0+1
rhvh-4.0-0.20170222.0
 +- rhvh-4.0-0.20170222.0+1
[root@localhost ~]# imgbase w
[INFO] You are on rhvh-4.0-0.20170222.0+1
[root@localhost ~]# ls -l /etc/ssh
total 276
-rw-r--r--. 1 root root     242153 Sep  6 09:30 moduli
-rw-r--r--. 1 root root       2208 Sep  6 09:30 ssh_config
-rw-------. 1 root root       4361 Sep  6 09:30 sshd_config
-rw-r-----. 1 root ssh_keys    227 Feb 23 07:07 ssh_host_ecdsa_key
-rw-r--r--. 1 root root        162 Feb 23 07:07 ssh_host_ecdsa_key.pub
-rw-r-----. 1 root ssh_keys    387 Feb 23 07:07 ssh_host_ed25519_key
-rw-r--r--. 1 root root         82 Feb 23 07:07 ssh_host_ed25519_key.pub
-rw-r-----. 1 root ssh_keys   1675 Feb 23 07:07 ssh_host_rsa_key
-rw-r--r--. 1 root root        382 Feb 23 07:07 ssh_host_rsa_key.pub
[root@localhost ~]# journalctl -u sshd.service
-- Logs begin at Thu 2017-02-23 08:15:52 MST, end at Thu 2017-02-23 11:07:55 MST. --
Feb 23 08:16:14 localhost.localdomain systemd[1]: Starting OpenSSH server daemon...
Feb 23 08:16:16 localhost.localdomain systemd[1]: PID file /var/run/sshd.pid not readable (yet?) after start.
Feb 23 08:16:16 localhost.localdomain sshd[1484]: Server listening on 0.0.0.0 port 22.
Feb 23 08:16:16 localhost.localdomain sshd[1484]: Server listening on :: port 22.
Feb 23 08:16:16 localhost.localdomain systemd[1]: Started OpenSSH server daemon.
Feb 23 10:44:07 localhost.localdomain sshd[3276]: Accepted password for root from 192.168.122.1 port 55050 ssh2

--------------------------------------------------------------------------------------

[root@localhost ~]# imgbase w
[INFO] You are on rhvh-4.0-0.20170222.0+1
[root@localhost ~]# imgbase layout
rhvh-4.0-0.20170222.0
 +- rhvh-4.0-0.20170222.0+1
[root@localhost ~]# ls -l /etc/ssh
total 276
-rw-r--r--. 1 root root     242153 Dec 20 09:27 moduli
-rw-r--r--. 1 root root       2208 Dec 20 09:27 ssh_config
-rw-------. 1 root root       4361 Dec 20 09:27 sshd_config
-rw-r-----. 1 root ssh_keys    227 Feb 23 08:25 ssh_host_ecdsa_key
-rw-r--r--. 1 root root        162 Feb 23 08:25 ssh_host_ecdsa_key.pub
-rw-r-----. 1 root ssh_keys    387 Feb 23 08:25 ssh_host_ed25519_key
-rw-r--r--. 1 root root         82 Feb 23 08:25 ssh_host_ed25519_key.pub
-rw-r-----. 1 root ssh_keys   1675 Feb 23 08:25 ssh_host_rsa_key
-rw-r--r--. 1 root root        382 Feb 23 08:25 ssh_host_rsa_key.pub
[root@localhost ~]# journalctl -u sshd.service
-- Logs begin at Thu 2017-02-23 08:24:54 MST, end at Thu 2017-02-23 11:08:16 MST. --
Feb 23 08:25:10 localhost.localdomain systemd[1]: Starting OpenSSH server daemon...
Feb 23 08:25:11 localhost.localdomain sshd[1481]: Server listening on 0.0.0.0 port 22.
Feb 23 08:25:11 localhost.localdomain sshd[1481]: Server listening on :: port 22.
Feb 23 08:25:11 localhost.localdomain systemd[1]: Started OpenSSH server daemon.
Feb 23 10:43:55 localhost.localdomain sshd[20818]: Accepted password for root from 192.168.122.1 port 34926 ssh2

(Originally by Ryan Barry)

Comment 5 rhev-integ 2017-02-27 13:24:04 UTC
Hi Ryan,
I have send you a email about this bug's detail log, and i will try to re-verify this problem on another machine, because we have reproduced this issue on our local machine 100%.

Jiawu
Thanks

(Originally by Jian Wu)

Comment 6 jianwu 2017-03-06 02:21:47 UTC
Hi,
I have verified this bug on redhat-virtualization-host-4.0-20170302.0.x86_64,

Version-Release number of selected component (if applicable):
Before upgrade:
redhat-virtualization-host-4.0-20161116.1.x86_64(el7.3)
After upgrade:
redhat-virtualization-host-4.0-20170302.0
imgbased-0.8.15-0.1.el7ev.noarch
kernel-3.10.0-514.10.2.el7.x86_64

Steps to test:
1. Install redhat-virtualization-host-4.0-20161116.1.x86_64(el7.3) via anaconda
2. Reboot and log into this system
3. Set local repo and run #yum update
4. Reboot into new build
5. Run #systemctl status sshd

Actual results:
After step 5, 
#systemctl status sshd
● sshd.service - OpenSSH server daemon
   Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2017-03-06 10:08:18 CST; 6min ago
     Docs: man:sshd(8)
           man:sshd_config(5)
  Process: 1483 ExecStart=/usr/sbin/sshd $OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 1540 (sshd)
   CGroup: /system.slice/sshd.service
           └─1540 /usr/sbin/sshd

So I think this bug is fixed, I will change status to VERIFIED.

Comment 9 errata-xmlrpc 2017-03-16 15:40:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2017-0549.html


Note You need to log in before you can comment on or make changes to this bug.