Bug 1049619

Summary: /etc/pki/ovirt-engine/cacert.conf is missing in 3.2 installation
Product: Red Hat Enterprise Virtualization Manager Reporter: Tomas Dosek <tdosek>
Component: rhevm-setup-pluginsAssignee: Yedidyah Bar David <didi>
Status: CLOSED ERRATA QA Contact: Jiri Belka <jbelka>
Severity: high Docs Contact: Jodi Biddle <jbiddle>
Priority: high    
Version: 3.3.0CC: acathrow, adahms, alonbl, bazulay, cfrancio, didi, iheim, lbopf, lyarwood, mkalinin, npatil, pablo.iranzo, pstehlik, Rhev-m-bugs, sbonazzo, scohen, sfolkwil, tdosek, yeylon
Target Milestone: ---Keywords: ZStream
Target Release: 3.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: integration
Fixed In Version: AV1 Doc Type: Bug Fix
Doc Text:
Previously, upgrading from Red Hat Enterprise Virtualization Manager version 3.1 to 3.2 and then from 3.2 to 3.3 would fail if cacert.conf was missing and cert.conf existed due to manual changes. Now, engine-setup takes this into account.
Story Points: ---
Clone Of:
: 1059242 (view as bug list) Environment:
Last Closed: 2014-06-09 13:31:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 902971, 1059242, 1078909, 1142926    
Attachments:
Description Flags
Upgrade log of the failure none

Description Tomas Dosek 2014-01-07 21:00:23 UTC
Description of problem:
Upgrade fails if cacert.conf is missing even in the case that we have all data needed to reconstruct it (especially after verification that database connection is ok).

The error displayed is:
[ ERROR ] Failed to execute stage 'Misc configuration': [Errno 2] No such file or directory: '/etc/pki/ovirt-engine/cacert.conf'

The only thing we need to do is to take the cacert.template and fill hostname and password for certificate from database.

Version-Release number of selected component (if applicable):
is29

How reproducible:
100 %

Steps to Reproduce:
1. Upgrade from 3.1 to 3.2
2. Try to upgrade to 3.3

Actual results:
Failure is shown about missing file (if manually created upgrade passes)

Expected results:
Upgrade creates the file using known data either from cert.conf or database

Additional info:
Attaching complete logs

Comment 1 Tomas Dosek 2014-01-07 21:02:58 UTC
Created attachment 846836 [details]
Upgrade log of the failure

Comment 2 Sandro Bonazzola 2014-01-08 09:17:22 UTC
I've installed rhevm 3.1, updated to 3.2 and then to 3.3 but cacert.conf was always there.
While I agree that cacert.conf can be recreated by setup, I'm not sure it's the right thing to do. Nothing in the upgrade process seems involved in its removal. 

I think that if cacert.conf is missing, system should be inspected because it's in an unstable state.
I propose as solution to check for cacert.conf existence in verification stage while upgrading from legacy 3.2.z and abort setup early telling the user to inspect the system and how to recreate cacert.conf manually if it has been deleted by mistake by the user.

Comment 3 Alon Bar-Lev 2014-01-08 09:21:39 UTC
(In reply to Sandro Bonazzola from comment #2)
> I propose as solution to check for cacert.conf existence in verification
> stage while upgrading from legacy 3.2.z and abort setup early telling the
> user to inspect the system and how to recreate cacert.conf manually if it
> has been deleted by mistake by the user.

Why only this file? we can verify any file out there... :)

This failure you got is just like verification - something wrong with the system and the root cause should be found before proceeding.

Comment 4 Yedidyah Bar David 2014-01-08 09:28:47 UTC
(In reply to Alon Bar-Lev from comment #3)
> (In reply to Sandro Bonazzola from comment #2)
> > I propose as solution to check for cacert.conf existence in verification
> > stage while upgrading from legacy 3.2.z and abort setup early telling the
> > user to inspect the system and how to recreate cacert.conf manually if it
> > has been deleted by mistake by the user.
> 
> Why only this file? we can verify any file out there... :)
> 
> This failure you got is just like verification - something wrong with the
> system and the root cause should be found before proceeding.

Any chance, then, to find out the root cause of the missing file? If it's merely a user mistake, I agree with Alon that we should do nothing.

Comment 5 Tomas Dosek 2014-01-08 09:30:36 UTC
Actually I blame rollback to delete the file. The upgrade failed on database ownership before and rolled back after that the validation of the file presence failed. 

I filed separate bug for that one: https://bugzilla.redhat.com/show_bug.cgi?id=1049622

Comment 6 Sandro Bonazzola 2014-01-08 09:44:23 UTC
(In reply to Tomas Dosek from comment #5)
> Actually I blame rollback to delete the file. The upgrade failed on database
> ownership before and rolled back after that the validation of the file
> presence failed. 
> 
> I filed separate bug for that one:
> https://bugzilla.redhat.com/show_bug.cgi?id=1049622

Trying to reproduce it too, but if cacert.conf is missing and you're using standard ports for apache, setup completes without errors. Error is raised only if you're using non standard ports on 3.2.z and cacert.conf is missing.

Comment 7 Alon Bar-Lev 2014-01-08 10:10:31 UTC
This may be related to bug#1003664.

As safeguard the process of copying old rhevm-3.0 pki artifacts is performed only if /etc/pki/ovirt-engine/cert.conf is missing.

Questions:

1. is it rhevm-3.0 upgraded machine?

2. do you have /etc/pki/rhevm-old?

3. what do you have in /etc/pki/ovirt-engine/cert.conf - as it should have been missing too.

Thanks!

Comment 8 Tomas Dosek 2014-01-08 10:18:00 UTC
1) It is

For 2 and 3 of comment 7: 
Nilesh could you please provide us with these?
I don't have direct access the the system so I need to ask GIS guys
to provide us with the input needed here.

Comment 10 Tomas Dosek 2014-01-09 14:00:05 UTC
I obtained reply from the GIS staff:

"I saw there are three questions asked in Private comment.

Questions:

1. is it rhevm-3.0 upgraded machine? ( yes it is upgraded from 3.0 to 3.1 and then 3.2 and now 3.3 beta)

2. do you have /etc/pki/rhevm-old? ( yes we have , i have also attached this in ticket)

3. what do you have in /etc/pki/ovirt-engine/cert.conf - as it should have been missing too. ( it is present on the server and i have attached the same in ticket also)

Please let me know if you require any further details for the same.

Thank you
shishir- "

Attaching the requested data right away.

Comment 25 Alon Bar-Lev 2014-01-18 19:58:04 UTC
(In reply to Tomas Dosek from comment #23)
> Complete /var/log/ovirt-engine is available

ovirt-engine-upgrade_2013_09_12_10_03_16.log
---
2013-09-12 10:38:28::DEBUG::upgrade_configs30::62::root:: PKI certificates were successfully restored from previous setup
<snip>
no reference for cert.conf
---
--> expected behavior (although incorrect).

ovirt-engine-upgrade_2014_01_07_12_13_19.log:
---
2014-01-07 12:25:39::DEBUG::rhevm-upgrade::684::root:: Checking legacy PKI upgrade failure
---
--> cert.conf exists as no "Found legacy PKI upgrade failure"

In between there is no rollback nor setup failure apart of early failure during yum prerequisites.

Unless there is more information, I still conclude that this cert.conf was added manually at some stage.

Thanks!

Comment 29 Alon Bar-Lev 2014-01-26 11:06:16 UTC
Although I have no idea why this happens, I could not get any flow in which cacert.conf is missing while cert.conf is not, I prepared a fix for that.

Comment 35 Jiri Belka 2014-03-07 14:23:29 UTC
ok, same reproduce steps as in https://bugzilla.redhat.com/show_bug.cgi?id=1059242#c3

rhevm-setup-3.4.0-0.3.master.el6ev.noarch

Comment 37 errata-xmlrpc 2014-06-09 13:31:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0653.html