Description of problem: After each host reboot, the ssh public key is regenerated, causing ssh to the host fail. Version-Release number of selected component (if applicable): rhev-hypervisor7-7.1-20150728.1.iso How reproducible: always Steps to Reproduce: 1.install node 2. ssh, answer yes to ssh prompt for saving the host udentification 3. reboot the hots 4. ssh again Actual results: ssh will complain that the host has another id then what is stored. Expected results: ssh should just work Additional info: to work around this, I persisted /etc/ssh/ssh_host_ecdsa_key and /config/etc/ssh/ssh_host_ecdsa_key.pub
Test version: rhev-hypervisor7-7.1-20150728.1 ovirt-node-3.2.3-14.el7 Only can reproduce this issue after _first_ time RHEV-H reboot. After the second time or more time reboot RHEV-H, the SSH fingerprint is not changed.
The attached patch does not fix the issue. 1. Install RHEV-H 7.1 2. Reboot and login to TUI 3. Drop to shell 4. grep ssh_host /config(files does _not_ list the files in the initial description
# rpm -qa ovirt-node ovirt-node-3.3.0-0.13.20151008git03eefb5.el7ev.noarch # cat /etc/redhat-release Red Hat Enterprise Virtualization Hypervisor release 7.2 (20151009.0.el7ev) Steps to Reproduce: 1. Installed RHEV-H 7.2 for 3.6.0 2. ssh, answer yes to ssh prompt for saving the host key. 3. restart the rhevh host 4. ssh host once more step 4: ssh works well.
The regression occurred between rhev-hypervisor7-7.1-20151009.0.el7ev and rhev-hypervisor7-7.2-20151025.0.el7ev. This issue is fixed on rhev-hypervisor7-7.1-20151009.0.el7ev. But in build rhev-hypervisor7-7.2-20151025.0.el7ev, this issue happened again, the SSH fingerprint was changed after _first_ time RHEV-H reboot. Steps to Reproduce: 1. installed rhevh 2. enable ssh 3. ssh via another host, answer yes to ssh prompt for saving the host udentification 4. reboot the rhevh host 5. ssh again After the second time or more time reboot RHEV-H, the SSH fingerprint is not changed.
Hi Cannot recreate it with rhev-hypervisor7-7.2-20151025.0.iso Can you please set me with a machine that it recreates
(In reply to Ying Cui from comment #6) > The regression occurred between rhev-hypervisor7-7.1-20151009.0.el7ev and > rhev-hypervisor7-7.2-20151025.0.el7ev. > > This issue is fixed on rhev-hypervisor7-7.1-20151009.0.el7ev. But in build > rhev-hypervisor7-7.2-20151025.0.el7ev, this issue happened again, the SSH > fingerprint was changed after _first_ time RHEV-H reboot. > > Steps to Reproduce: > 1. installed rhevh > 2. enable ssh > 3. ssh via another host, answer yes to ssh prompt for saving the host > udentification > 4. reboot the rhevh host > 5. ssh again > > After the second time or more time reboot RHEV-H, the SSH fingerprint is not > changed. I couldn't reproduce this report yet. However, testing some upgrade scenarios and I have found this semodule AVC and might be related. semodule: Add rule ro AVC sshd_keygen_t plymouth_exec_t https://gerrit.ovirt.org/#/c/48098/( @Ying, I will try more times to reproduce it. Is it 100% of time for you? If yes, if you add selinux=0 in kargs is it solve the issue? Thanks!
(In reply to Douglas Schilling Landgraf from comment #10) > (In reply to Ying Cui from comment #6) > > The regression occurred between rhev-hypervisor7-7.1-20151009.0.el7ev and > > rhev-hypervisor7-7.2-20151025.0.el7ev. > > > > This issue is fixed on rhev-hypervisor7-7.1-20151009.0.el7ev. But in build > > rhev-hypervisor7-7.2-20151025.0.el7ev, this issue happened again, the SSH > > fingerprint was changed after _first_ time RHEV-H reboot. > > > > Steps to Reproduce: > > 1. installed rhevh > > 2. enable ssh > > 3. ssh via another host, answer yes to ssh prompt for saving the host > > udentification > > 4. reboot the rhevh host > > 5. ssh again > > > > After the second time or more time reboot RHEV-H, the SSH fingerprint is not > > changed. > > I couldn't reproduce this report yet. However, testing some upgrade > scenarios and I have found this semodule AVC and might be related. > > semodule: Add rule ro AVC sshd_keygen_t plymouth_exec_t > https://gerrit.ovirt.org/#/c/48098/( > > @Ying, I will try more times to reproduce it. Is it 100% of time for you? > If yes, if you add selinux=0 in kargs is it solve the issue? @Douglas, Yes, it is 100% reproduce on QE's machines. Steps to Reproduce: 1. installed rhevh 2. enable ssh 3. ssh via another host, answer yes to ssh prompt for saving the host udentification 4. reboot the rhevh host 5. ssh again --- # issue happen, the SSH fingerprint is changed, need resave SSH fingerprint 6. reboot the rhevh host once more 7. ssh again --- # the SSH fingerprint is not changed this issue only happen after the first time reboot rhevh. After the second time or more time reboot RHEV-H, the SSH fingerprint is not changed.
Tested RHEV-H 7.2 20151029.0 for rhev 3.5.6, this issue still here. So this bug affect rhev 3.5.z as well. And this issue was fixed in RHEVH 3.5.5(rhev-hypervisor7-7.1-20150917.0.iso and rhev-hypervisor6-6.7-20150917.0.iso) yet. So the regression happen on RHEV-H 3.5.6.
Please set me up with the environment containing install from comment 9
Hi, I couldn't reproduce the report but below my findings with the last logs from 10.66.9.205. First, it seems a different root cause then the original report. (If confirmed, better open a different bug, to avoid mix issues) Based in the event logs: #1 - sshd-keygen daemon started (/var/log/messages) ================================================= Nov 6 11:24:39 localhost sshd-keygen: Generating SSH2 RSA host key: [ OK ] Nov 6 11:24:39 localhost sshd-keygen: Generating SSH2 ECDSA host key: [ OK ] Nov 6 11:24:39 localhost sshd-keygen: Generating SSH2 ED25519 host key: [ OK ] Nov 6 11:24:39 localhost systemd: Started OpenSSH Server Key Generation. #2 - Now in the same messages file, we see an AVC from selinux preventing/denying sshd-keygen daemon. *Please note* this AVC was not int audit/audit.log. /var/log/messages ========================================= Nov 6 11:27:09 localhost sshd-keygen: Generating SSH2 RSA host key: [FAILED] Nov 6 11:27:09 localhost kernel: type=1400 audit(1446809229.521:41): avc: denied { execute } for pid=1150 comm="sshd-keygen" name="plymouth" dev="dm-0" ino=4492 scontext=system_u:system_r:sshd_keygen_t:s0 tcontext=unconfined_u:object_r:plymouth_exec_t:s0 tclass=file Nov 6 11:27:09 localhost kernel: type=1300 audit(1446809229.521:41): arch=c000003e syscall=269 success=no exit=-13 a0=ffffffffffffff9c a1=1e500c0 a2=1 a3=7ffc2297fb30 items=0 ppid=1 pid=1150 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="sshd-keygen" exe="/usr/bin/bash" subj=system_u:system_r:sshd_keygen_t:s0 key=(null) Nov 6 11:27:09 localhost systemd: sshd-keygen.service: main process exited, code=exited, status=1/FAILURE Nov 6 11:27:09 localhost systemd: Failed to start OpenSSH Server Key Generation. Nov 6 11:27:09 localhost systemd: Unit sshd-keygen.service entered failed state. #3 - Later, looks like ssh daemon is enabled and restarted via TUI, generating the ssh keys. ================================================================ * (/var/log/ovirt-node.log) <snip> 2015-11-06 11:29:29,299 WARNING File "/etc/ssh/sshd_config" had already been persisted </snip> * (in /var/log/messages at the same time): <snip> Nov 6 11:29:29 localhost systemd: Started OpenSSH Server Key Generation. Nov 6 11:29:29 localhost systemd: Starting OpenSSH server daemon... Nov 6 11:29:29 localhost systemd: Started OpenSSH server daemon. Nov 6 11:29:29 localhost systemd: Stopping OpenSSH server daemon... Nov 6 11:29:29 localhost systemd: Stopping OpenSSH Server Key Generation... Nov 6 11:29:29 localhost systemd: Started OpenSSH Server Key Generation. Nov 6 11:29:29 localhost systemd: Starting OpenSSH server daemon... Nov 6 11:29:29 localhost systemd: Started OpenSSH server daemon. </snip> At the same time keys got generated. # ls -la /etc/ssh/ total 271 drwxr-xr-x. 2 root root 220 Nov 6 11:29 . drwxr-xr-x. 104 root root 4560 Nov 6 11:29 .. -rw-r--r--. 1 root root 242153 May 12 19:41 moduli -rw-r--r--. 1 root root 2233 Nov 5 14:25 ssh_config -rw-------. 1 root root 4414 Nov 6 11:29 sshd_config -rw-r-----. 1 root ssh_keys 227 Nov 6 11:29 ssh_host_ecdsa_key -rw-r--r--. 1 root root 162 Nov 6 11:29 ssh_host_ecdsa_key.pub -rw-r-----. 1 root ssh_keys 387 Nov 6 11:29 ssh_host_ed25519_key -rw-r--r--. 1 root root 82 Nov 6 11:29 ssh_host_ed25519_key.pub -rw-r-----. 1 root ssh_keys 1679 Nov 6 11:24 ssh_host_rsa_key -rw-r--r--. 1 root root 382 Nov 6 11:24 ssh_host_rsa_key.pub A possible explanation for the report from Ying in comment#12 ================================================================ First boot selinux prevent sshd-keygen to generate the ssh keys and our init/boot script couldn't persist the keys in /etc/ssh. Later, ssh service enabled/restarted via TUI the keys got generated but not persisted. So, user could ssh to machine but the /etc/ssh keys were not persisted and next reboot the files were lost and regenerated.
To confirm DOuglas findings in comment 20, Ying, could you try to install in permissive mode and see if you can reproduce the bug there?
(In reply to Fabian Deutsch from comment #21) > To confirm DOuglas findings in comment 20, Ying, could you try to install in > permissive mode and see if you can reproduce the bug there? @Douglas, yes, I tested with enforcing=0, not help. still happened the issue. see comment 23, the issue already fixed in ovirt-node-iso-3.6-0.999.201511081037.el7.centos.iso, if you think that is different root cause for the same reproduce steps, we can split a new bug, do we need?
(In reply to Ying Cui from comment #24) > (In reply to Fabian Deutsch from comment #21) > > To confirm DOuglas findings in comment 20, Ying, could you try to install in > > permissive mode and see if you can reproduce the bug there? > > @Douglas, yes, I tested with enforcing=0, not help. still happened the issue. > Thanks for the testing. > see comment 23, the issue already fixed in > ovirt-node-iso-3.6-0.999.201511081037.el7.centos.iso, if you think that is > different root cause for the same reproduce steps, we can split a new bug, > do we need? It's not needed at moment, I will discuss with the others developers if we should include the patch or not in rhev-h meetings. If requires, we can use the same bug as it's related topic (ssh-keygen).
The last test RHEV-H 7.2 for 3.6 beta 1 build (rhev-hypervisor7-7.2-20151112.1.el7ev) did not include this bug fix, both ovirt-node and rhevh build with this bug fix is not built in brew yet. We have to move this ON_QA bug to MODIFIED for pending on available downstream build to QE. # rpm -qa ovirt-node ovirt-node-3.6.0-0.20.20151103git3d3779a.el7ev.noarch # cat /etc/redhat-release Red Hat Enterprise Virtualization Hypervisor (Beta) release 7.2 (20151112.1.el7ev)
I can‘ t verified this bug，please provide the Fixed In Version
Please provide the Fixed In Version
Test version: RHEV-H 7.2-20151210.1.el7ev ovirt-node-3.6.0-0.24.20151209gitc0fa931.el7ev.noarch Steps to Reproduce: 1. Install node 2. Ssh, answer yes to ssh prompt for saving the host udentification 3. Reboot the hots 4. Ssh again Test result: After step4, ssh should just work So the bug is fixed, change bug status to VERIFIED.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-0378.html