Bug 1051001 - [RHEVM-SETUP] - remote db configuration. unclear state after roll-back
Summary: [RHEVM-SETUP] - remote db configuration. unclear state after roll-back
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine-setup
Version: 3.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: 3.3.0
Assignee: Sandro Bonazzola
QA Contact: Pavel Stehlik
URL:
Whiteboard:
Depends On: 1049622
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-01-09 14:11 UTC by Barak Dagan
Modified: 2014-07-13 23:20 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-01-09 15:49:54 UTC
oVirt Team: ---
Target Upstream Version:


Attachments (Terms of Use)
engine + server + setup logs (102.29 KB, application/x-compressed-tar)
2014-01-09 14:11 UTC, Barak Dagan
no flags Details

Description Barak Dagan 2014-01-09 14:11:53 UTC
Created attachment 847675 [details]
engine + server + setup logs

Description of problem:

Rollback of upgrade 3.2 -> 3.3, doesn't work correctly (at leaset with remote db configuration):

Created by using non valid db upgarde script
(/usr/share/ovirt-engine/dbscripts/upgrade/03_02_0470_test_me.sql which contains some randomized / meaningless strings).

1) Engine's rollback doesn't work correctly:

# rpm -q rhevm
rhevm-3.3.0-0.43.el6ev.noarch

web admin: 
"Service Temporarily Unavailable

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later."

engine.log:
==========
2014-01-09 14:56:23,379 ERROR [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 40) Error in getting DB connection. The database is inaccessible. Original exception is: DataAccessResourceFailureException: Error retreiving database metadata; nested exception is org.springframework.jdbc.support.MetaDataAccessException: Could not get Connection for extracting meta data; nested exception is org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource

server.log:
==========
2014-01-09 14:56:24,912 ERROR [org.jboss.as] (Controller Boot Thread) JBAS015875: JBoss EAP 6.2.0.GA (AS 7.3.0.Final-redhat-14) started (with errors) in 313125ms - Started 622 of 679 services (8 services failed or missing dependencies, 48 services are passive or on-demand)


Version-Release number of selected component (if applicable):
3.2.5.1 -> is30

How reproducible:
100%

Steps to Reproduce:
1. Install 3.2.5 engine
2. Add another fake db upgarde script in order to fail the upgarde process
3. Perform upgrade (it will fail and rollback)
4. remove fake upagrde script (to leave clean setup)
5. Start ovirt-engine service

Actual results:
Engine service starts, but engine doesn't respone

Expected results:
Engine service starts and functional.

Additional info:
This issues leaves the system in non functional / non upgarded state.

Comment 1 Sandro Bonazzola 2014-01-09 15:28:11 UTC
(In reply to Barak Dagan from comment #0)

> Created by using non valid db upgarde script
> (/usr/share/ovirt-engine/dbscripts/upgrade/03_02_0470_test_me.sql which
> contains some randomized / meaningless strings).

I'll try to reproduce but I'm not sure this is really a valid test, altering the code that perform the schema upgrade.

Comment 2 Alon Bar-Lev 2014-01-09 15:32:53 UTC
Correct, closing.

Comment 3 Yedidyah Bar David 2014-01-09 15:42:20 UTC
(In reply to Alon Bar-Lev from comment #2)
> Correct, closing.

I do not agree. However:

1. The log says, among other things:

2014-01-08 17:27:48 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:215 DIALOG:SEND                 Setup will not be able to rollback new packages in case of a failure, because installed ones were not found in enabled repositories.
2014-01-08 17:27:48 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:215 DIALOG:SEND                 Do you want to abort Setup? (Yes, No) [Yes]: 
2014-01-08 17:27:56 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:215 DIALOG:RECEIVE    no

so it could not rollback.

2. I did a similar test today and it passed well. The only issue I had was that the service was not started, and I had to manually start it and then restart httpd.

Barak - please try to reproduce with both repos available so that it can rollback. Thanks.

Comment 4 Sandro Bonazzola 2014-01-09 15:49:54 UTC
(In reply to Yedidyah Bar David from comment #3)

> 2014-01-08 17:27:48 DEBUG otopi.plugins.otopi.dialog.human
> dialog.__logString:215 DIALOG:SEND                 Setup will not be able to
> rollback new packages in case of a failure, because installed ones were not
> found in enabled repositories.
> 2014-01-08 17:27:48 DEBUG otopi.plugins.otopi.dialog.human
> dialog.__logString:215 DIALOG:SEND                 Do you want to abort
> Setup? (Yes, No) [Yes]: 
> 2014-01-08 17:27:56 DEBUG otopi.plugins.otopi.dialog.human
> dialog.__logString:215 DIALOG:RECEIVE    no
> 
> so it could not rollback.

So I would have closed as NOTABUG instead of WONTFIX, not moved to NEW.
Added a needinfo on Barak as per didi question, but moving to NOTABUG for now.

Comment 5 Barak Dagan 2014-01-12 07:24:59 UTC
I do not agree. Since setup doesn't require to have both repos, and allow to continue with such use case, I'd expect it not to start rollback scenario in case of failure. The current situation is having DB of older version but code of a newer one, if that's not a bug ....

Comment 6 Yedidyah Bar David 2014-01-12 07:47:44 UTC
(In reply to Barak Dagan from comment #5)
> I do not agree. Since setup doesn't require to have both repos, and allow to
> continue with such use case, I'd expect it not to start rollback scenario in
> case of failure. The current situation is having DB of older version but
> code of a newer one, if that's not a bug ....

Well, if you consider that a bug, please:
1. Reopen, probably with lower urgency (zstream, 3.4, etc.)
2. Set a more accurate Summary line, such as "If setup fails and can't rollback packages, ..."
3. State clearly what you expect to happen in such a case. In particular, your current Expected results "Engine service starts and functional" are unrealistic. If there is a problem and setup can't rollback to previous packages, it will leave the system broken. How broken exactly - database partially updated, database rolled back to previous content, whatever - is a detail, imo an insignificant one.

I do not see any relation between current summary and content - perhaps you meant to open another bug?

Comment 7 Barak Dagan 2014-01-12 08:08:37 UTC
(In reply to Yedidyah Bar David from comment #6)
> (In reply to Barak Dagan from comment #5)
> > I do not agree. Since setup doesn't require to have both repos, and allow to
> > continue with such use case, I'd expect it not to start rollback scenario in
> > case of failure. The current situation is having DB of older version but
> > code of a newer one, if that's not a bug ....
> 
> Well, if you consider that a bug, please:
> 1. Reopen, probably with lower urgency (zstream, 3.4, etc.)
> 2. Set a more accurate Summary line, such as "If setup fails and can't
> rollback packages, ..."
> 3. State clearly what you expect to happen in such a case. In particular,
> your current Expected results "Engine service starts and functional" are
> unrealistic. If there is a problem and setup can't rollback to previous
> packages, it will leave the system broken. How broken exactly - database
> partially updated, database rolled back to previous content, whatever - is a
> detail, imo an insignificant one.
> 
> I do not see any relation between current summary and content - perhaps you
> meant to open another bug?

I agree about the summary, was copied, unintentionally, from a previous one - so I fix that one.
As for the expectation, was before you revealed the cause for this issue.


Note You need to log in before you can comment on or make changes to this bug.