Bug 1270881

Summary: [engine][host reinstall] 'Reinstall'ing with password fails because of ssh fingerprint
Product: [oVirt] ovirt-engine Reporter: Jiri Belka <jbelka>
Component: Host-DeployAssignee: Moti Asayag <masayag>
Status: CLOSED NOTABUG QA Contact: Pavel Stehlik <pstehlik>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.6.0.1CC: alonbl, bugs, masayag, oourfali, ylavi
Target Milestone: ovirt-3.6.1Keywords: Reopened
Target Release: ---Flags: oourfali: ovirt-3.6.z?
oourfali: ovirt-4.0.0?
rule-engine: planning_ack?
rule-engine: devel_ack+
rule-engine: testing_ack?
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: infra
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-10-26 10:52:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
engine.log none

Description Jiri Belka 2015-10-12 15:12:55 UTC
Created attachment 1082032 [details]
engine.log

Description of problem:

If you click 'Reinstall' on a host (being in maintenance) which got different SSH server keys (OS reinstall) and you input password, then the action will fail as engine compares already known server's ssh server key fingerprint with actual ssh server key fingerprint.

This seems odd especially when you typed password and you cannot modify ssh fingerprint field in 'Reinstall' (in fact it is 'Install host') dialog.

(This flow seems to be quicker than Remove and Add Host and thus I suppose more people could try this.)

Either when password is used ssh server fingerprint should be totally ignore or there should be checkbox or warning about changed ssh key fingerprint.


----%----
2015-10-12 16:48:23,484 INFO  [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-27) [389c4e5d] Running command: InstallVdsInternalCommand(oVirtIsoFile = null, IsRein
stallOrUpgrade = true, AuthMethod = Password, NetworkMappings = null, VdsStaticData = null, vds = Host[null], OverrideFirewall = true, ActivateHost = true, RebootAfterInstallation = true, NetworkProviderId = null
, EnableSerialConsole = true, VdsId = 1797ba0c-f63d-490c-929a-31a834106e3c, RunSilent = false) internal: true. Entities affected :  ID: 1797ba0c-f63d-490c-929a-31a834106e3c Type: VDS
...
2015-10-12 16:48:23,580 DEBUG [org.ovirt.engine.core.uutils.ssh.OpenSSHUtils] (org.ovirt.thread.pool-7-thread-27) [389c4e5d] Fingerprint: SHA256:UeuopKmqWgyLDLvFtkhQJVENUC1ZYGhTdy48WP1buWw
2015-10-12 16:48:23,580 DEBUG [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-7-thread-27) [389c4e5d] Could not connect to host 'root.lab.eng.brq.redhat.com'
2015-10-12 16:48:23,580 DEBUG [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-7-thread-27) [389c4e5d] Exception: java.security.GeneralSecurityException: Invalid fingerprint SHA256:UeuopKmqWgyL
DLvFtkhQJVENUC1ZYGhTdy48WP1buWw, expected SHA256:ag62ZttItQRGs07saArsiwYT3nmkJ+1qRxMWbBcDAaI
...
----%----

Version-Release number of selected component (if applicable):
rhevm-backend-3.6.0-0.18.el6.noarch

How reproducible:
100%

Steps to Reproduce:
1. add host into engine
2. put the host into maintenance
3. change ssh server keys (reinstall OS would do it too)
4. click 'Reinstall' of the host

Actual results:
failure, mismatched ssh fingerprint

Expected results:
imo should work (better with a warning)

Additional info:

Comment 1 Alon Bar-Lev 2015-10-18 08:23:56 UTC
You can modify the fingerprint in edit host.
It is similar to what openssh has:
1. first install you can fetch fingerprint / accept whatever remote has.
2. after that only manual edit to reduce mim issues.

Comment 2 Red Hat Bugzilla Rules Engine 2015-10-19 10:50:35 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 3 Moti Asayag 2015-10-20 05:45:53 UTC
Based on explanation in Comment 1 - moving to ON_QA, with the following steps to reproduce:

1. add host into engine
2. put the host into maintenance
3. change ssh server keys (reinstall OS would do it too)
4. Edit host and set the new fingerprint
5. click 'Reinstall' of the host

Comment 4 Yaniv Lavi 2015-10-21 11:13:27 UTC
What is the target for this?
What version fixed this issue?

Comment 5 Alon Bar-Lev 2015-10-21 11:14:28 UTC
(In reply to Yaniv Dary from comment #4)
> What is the target for this?
> What version fixed this issue?

no issue, should have been closed as NOTABUG.

Comment 6 Jiri Belka 2015-10-26 10:06:38 UTC
(In reply to Moti Asayag from comment #3)
> Based on explanation in Comment 1 - moving to ON_QA, with the following
> steps to reproduce:
> 
> 1. add host into engine
> 2. put the host into maintenance
> 3. change ssh server keys (reinstall OS would do it too)
> 4. Edit host and set the new fingerprint

It is not possible to edit/change 'SSH PublicKey' area in 'Install Host' (Reinstall action) dialog.

It should be possible. IMO direct editing of DB is not convenient.

> 5. click 'Reinstall' of the host

Comment 7 Jiri Belka 2015-10-26 10:08:06 UTC
So what I meant is, that editing should be possible also in 'Install Host' dialog.

Comment 8 Moti Asayag 2015-10-26 10:34:22 UTC
(In reply to Jiri Belka from comment #6)
> (In reply to Moti Asayag from comment #3)
> > Based on explanation in Comment 1 - moving to ON_QA, with the following
> > steps to reproduce:
> > 
> > 1. add host into engine
> > 2. put the host into maintenance
> > 3. change ssh server keys (reinstall OS would do it too)
> > 4. Edit host and set the new fingerprint
> 
> It is not possible to edit/change 'SSH PublicKey' area in 'Install Host'
> (Reinstall action) dialog.
> 

Why do you need to edit that field on the host ? This is the engine's public key.
If you wish to use this method, you need to add the engine's public key to the server's authorized_ids. Else, provide the password when reinstalling a host.

> It should be possible. IMO direct editing of DB is not convenient.
> 
> > 5. click 'Reinstall' of the host

Comment 9 Jiri Belka 2015-10-26 10:52:00 UTC
> > > 4. Edit host and set the new fingerprint
> > 
> > It is not possible to edit/change 'SSH PublicKey' area in 'Install Host'
> > (Reinstall action) dialog.
> > 
> 
> Why do you need to edit that field on the host ? This is the engine's public
> key.
> If you wish to use this method, you need to add the engine's public key to
> the server's authorized_ids. Else, provide the password when reinstalling a
> host.

Ooops, pebkac issue. I was confused and I've thought it is remote server ssh fingerprint.

So, it's not a bug.

Anyway, to clarify some thing here:

- What is that value in Edit host dialog for 'SSH Fingerprint'?

  SHA256:vGli07HKsbOURlPPe/Ksq2JKwgXA0hjtU9A+rXeyHFo

Because it does not look like ssh fingerprint...

[root@dell-r210ii-04 ~]# ssh-keygen -l -f /etc/ssh/ssh_host_ecdsa_key.pub
256 4b:df:8d:4d:e8:8e:09:b6:f9:72:09:2e:d4:62:4d:df   (ECDSA)
[root@dell-r210ii-04 ~]# ssh-keygen -l -f /etc/ssh/ssh_host_ed25519_key.pub 
256 1d:46:d1:1a:00:be:43:f8:c5:d0:2d:35:58:d2:e1:56   (ED25519)
[root@dell-r210ii-04 ~]# ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key.pub 
2048 a7:09:75:d9:f4:3b:9d:d7:3d:1a:a4:63:93:16:c3:3c   (RSA)

Comment 10 Alon Bar-Lev 2015-10-26 11:11:53 UTC
Since SHA algorithm that is being used for digest, the method of displaying the fingerprint is algorithm:base64, this makes result more readable and portable, as hash is now specify within the output.

See:

$ ssh root.0.71
The authenticity of host '10.35.0.71 (10.35.0.71)' can't be established.
ECDSA key fingerprint is SHA256:G2GAthRObSnT13jBb7bKl2P0Tf8ucuEqXaYJOdfqHUA.
Are you sure you want to continue connecting (yes/no)? 

echo "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdnh7kxq6sBQibDAEvoCxzeOqXaUGOWcReWFOuzEXCD2QrzD4k88MSLX1axkql0td1dzA4NFwjac1k8vs90iRRd0lMJq/+1Pw/GDX1Kn2ppZ+nbzEAMOIeRwnCgBKqcki7cUmbfr2lzztvobD0ljjyuQCsVbjI0XweUYDGWCv/5xl8V1SYAzlhB52pTOOCW7jRg4T2NFNIVAYDs3JdXOWbFO+ByzW6ooLXB0A0IdLoK81Uz+wYOfObOiH29RoH669YfUbzBcX2lz902S9ekW6aj6TEWtaN9M+698ZlNLerCkEhUjDUQAsY6wczf9ybb7a8Mj5mAagV31WbmcUmF90x" | ssh-keygen -f /proc/self/fd/0 -E sha256 -l
2048 SHA256:wQDSSmlW4caaBxRGMq83BlwCHZrEmR2P1JTW0XW90o0 /proc/self/fd/0 (RSA)

MD5 hashes should not be used any more.