Bug 1124737

Summary: Removing ovirt-engine and installing it again invalidate all hosts certificates
Product: [Retired] oVirt Reporter: Nir Soffer <nsoffer>
Component: ovirt-engine-installerAssignee: Yedidyah Bar David <didi>
Status: CLOSED NOTABUG QA Contact: Pavel Stehlik <pstehlik>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.5CC: acathrow, amureini, didi, ecohen, gklein, iheim, nsoffer, sbonazzo, yeylon
Target Milestone: ---Keywords: Triaged
Target Release: 3.5.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: integration
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-07-30 13:31:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Nir Soffer 2014-07-30 08:14:00 UTC
Description of problem:

After removing ovirt-engine and installing a new version, all hosts becomes non-responsive because of SSL Error in vdsm.

Reinstalling the hosts from engine fix this issue. 

Version-Release number of selected component (if applicable):
oVirt Engine Version: 3.5.0-0.0.master.20140722232056.git8e1babc.fc19

How reproducible:
Happened once, I remember same issue from previous installation.

Steps to Reproduce:
1. Have a working ovirt-engine setup
2. yum remove ovirt-engine\*
3. yum install ovirt-engine
4. engine-setup (accept defaults)

Actual results:
Activating a host, host becomes non-operational, because of SSL error in vdsm side.

Expected results:
Activating host should succeed

Additional info:
Looks like engine does not use the old keys, invalidating hosts using keys generated by the previous installation.

Workaround:
Reinstall all hosts

Unfortunately, vdsm logs with the SSL error are not available.

Comment 1 Yedidyah Bar David 2014-07-30 10:11:54 UTC
(In reply to Nir Soffer from comment #0)
> Description of problem:
> 
> After removing ovirt-engine and installing a new version, all hosts becomes
> non-responsive because of SSL Error in vdsm.
> 
> Reinstalling the hosts from engine fix this issue. 
> 
> Version-Release number of selected component (if applicable):
> oVirt Engine Version: 3.5.0-0.0.master.20140722232056.git8e1babc.fc19
> 
> How reproducible:
> Happened once, I remember same issue from previous installation.
> 
> Steps to Reproduce:
> 1. Have a working ovirt-engine setup
> 2. yum remove ovirt-engine\*
> 3. yum install ovirt-engine
> 4. engine-setup (accept defaults)
> 
> Actual results:
> Activating a host, host becomes non-operational, because of SSL error in
> vdsm side.
> 
> Expected results:
> Activating host should succeed
> 
> Additional info:
> Looks like engine does not use the old keys

You seem to imply here that it can. Can it? Do you see them?

AFAIU they are deleted by rpm (%preun/%postun).
This addition to the spec file was done shortly before the introduction of engine-cleanup, and it seems to me that it was first intended to try and cleanup at package removal, then a decision was made to add engine-cleanup, which also does that, but the change to the spec file was never reverted, probably because no-one really cared.

Note that given:
1. yum install ovirt-engine
2. engine-setup
reverting this change to the spec file will essentially make this:
3. yum remove ovirt-engine
4. yum install ovirt-engine
a no-op, but only if installing the exact same version. The correct way to upgrade to a newer version is to run engine-setup with an updated repo (as applicable). Since without reverting it (3.)+(4.) makes the engine non-functional, users have to run engine-setup, which also upgrades the db. So if we revert it, we probably want to force such an update using other means, not sure which - perhaps compare the installed version with the one saved in the db on engine startup and alert/abort if not same. Perhaps this is already done, didn't check.

Comment 2 Yedidyah Bar David 2014-07-30 10:26:32 UTC
Any reason, btw, to not simply run 'engine-setup', which is the correct way to upgrade?

Comment 3 Nir Soffer 2014-07-30 10:30:02 UTC
(In reply to Yedidyah Bar David from comment #2)
> Any reason, btw, to not simply run 'engine-setup', which is the correct way
> to upgrade?

I was running nightly version (3.6.0), did not want to created noise by testing an upgrade that a real user is unlikely to do.

In this way I was still using database from the future, but I don't see a way to avoid this and keep my complex setup as is.

Comment 4 Yedidyah Bar David 2014-07-30 11:25:44 UTC
(In reply to Nir Soffer from comment #3)
> (In reply to Yedidyah Bar David from comment #2)
> > Any reason, btw, to not simply run 'engine-setup', which is the correct way
> > to upgrade?
> 
> I was running nightly version (3.6.0), did not want to created noise by
> testing an upgrade that a real user is unlikely to do.
> 
> In this way I was still using database from the future, but I don't see a
> way to avoid this and keep my complex setup as is.

Very well, so you'll need to rephrase the summary to make it look like a bug...

For you specific case you could simply copy pki from the backup - it's in
/etc/pki/ovirt-engine-backups/ovirt-engine-TIMESTAMP .

Comment 5 Nir Soffer 2014-07-30 11:43:17 UTC
(In reply to Yedidyah Bar David from comment #4)
> (In reply to Nir Soffer from comment #3)
> > (In reply to Yedidyah Bar David from comment #2)
> Very well, so you'll need to rephrase the summary to make it look like a
> bug...

Feel free to improve the summary, as you probably understand the underlying issue better.

Comment 6 Yedidyah Bar David 2014-07-30 13:31:18 UTC
(In reply to Nir Soffer from comment #5)
> (In reply to Yedidyah Bar David from comment #4)
> > (In reply to Nir Soffer from comment #3)
> > > (In reply to Yedidyah Bar David from comment #2)
> > Very well, so you'll need to rephrase the summary to make it look like a
> > bug...
> 
> Feel free to improve the summary, as you probably understand the underlying
> issue better.

Sorry I wasn't clear. I do not think this is a bug.

The correct way to upgrade the engine is to run 'engine-setup'.

The correct way to remove it completely is to run 'engine-cleanup && yum remove ovirt-engine'.

Other flows might work, might work partially or not in the exact expected way, but are not officially supported. This includes, among other things, 'engine-setup && yum remove ovirt-engine && yum install ovirt-engine', whether or not the removed and installed packages are of the same version.

"Fixing" 'yum remove ovirt-engine' to not remove pki is very simple, but will not make the above recommended. What you specifically wanted to do might be legitimate for a development environment, but is obviously unsupportable generally.

Closing. Please reopen if relevant.

Comment 7 Yedidyah Bar David 2014-08-03 06:18:14 UTC
*** Bug 1058646 has been marked as a duplicate of this bug. ***