Bug 949694
| Summary: | rhevm-upgrade is failing between si26.4 to si27.4 (3.1.3) in async task cleanup | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Idith Tal-Kohen <italkohe> | ||||||||||||
| Component: | ovirt-engine-setup | Assignee: | Alex Lourie <alourie> | ||||||||||||
| Status: | CLOSED ERRATA | QA Contact: | Ilanit Stein <istein> | ||||||||||||
| Severity: | high | Docs Contact: | |||||||||||||
| Priority: | urgent | ||||||||||||||
| Version: | 3.1.3 | CC: | acathrow, alourie, bazulay, chetan, cpelland, dyasny, emesika, hateya, iheim, mgoldboi, Rhev-m-bugs, sgrinber, ykaul, yzaslavs | ||||||||||||
| Target Milestone: | --- | Keywords: | ZStream | ||||||||||||
| Target Release: | 3.1.4 | ||||||||||||||
| Hardware: | x86_64 | ||||||||||||||
| OS: | Linux | ||||||||||||||
| Whiteboard: | infra | ||||||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||||||
| Doc Text: |
Upgrading the Manager failed when there were leftover compensation entries during the async tasks cleanup. These entries caused the backend to abort when trying to activate the compensation procedure on system startup. The taskcleaner utility now uses the -A flag, which enables rhevm-upgrade to remove all compensation data from the business_entity_snapshot table during upgrade.
|
Story Points: | --- | ||||||||||||
| Clone Of: | 921201 | Environment: | |||||||||||||
| Last Closed: | 2013-05-01 18:35:47 UTC | Type: | Bug | ||||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||||
| Documentation: | --- | CRM: | |||||||||||||
| Verified Versions: | Category: | --- | |||||||||||||
| oVirt Team: | Infra | RHEL 7.3 requirements from Atomic Host: | |||||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||||
| Embargoed: | |||||||||||||||
| Bug Depends On: | 921201, 921202 | ||||||||||||||
| Bug Blocks: | |||||||||||||||
| Attachments: |
|
||||||||||||||
|
Comment 1
Alex Lourie
2013-04-18 09:40:18 UTC
https://gerrit.eng.lab.tlv.redhat.com/gitweb?p=ovirt-engine.git;a=commit;h=ae3b3c9e03b0de92454edd58d3f83e680c9292ae - fix for utility script + stored procedures. Checked si26.1 upgrade to si27.4 (si26.4 do not exist).
Scenrio:
=======
On si26.1:
1. Created a VM, and then created a template (started aync task).
2. While template creation running, stopped ovirt-engine service
3. Run "yum update rhevm-setup" (to si27.4)
4. Run "rhevm-upgrade" (to si27.4)
Results:
=======
The output of the rhevm-upgrade indeed different. No repeated question regarding stop async tasks, but,
Got an exception Errors on engine.log during clear tasks by upgrade process.
On GUI, VM remained in lock state (At least for 30 min, that I've checked).
Restart vdsmd service, and then restart ovirt engine didn't change the VM lock on GUI.
Exception Errors on engine.log during clear tasks by upgrade process:
=====================================================================
2013-04-24 02:29:07,529 ERROR [org.ovirt.engine.core.bll.EntityAsyncTask] (pool-3-thread-27) EntityAsyncTask::EndCommandAction [within thread]: EndAction for action type AddVmTemplate threw an exception: org.springframework.dao.DataIntegrityViolationException: CallableStatementCallback; SQL [{call deletevmtemplates(?)}]; ERROR: update or delete on table "vm_static" violates foreign key constraint "vm_templates_vm_static" on table "vm_static"
Detail: Key (vm_guid)=(00000000-0000-0000-0000-000000000000) is still referenced from table "vm_static".
Where: SQL statement "DELETE FROM vm_static WHERE vm_guid = $1 AND entity_type = 'TEMPLATE'"
PL/pgSQL function "deletevmtemplates" line 7 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: update or delete on table "vm_static" violates foreign key constraint "vm_templates_vm_static" on table "vm_static"
Detail: Key (vm_guid)=(00000000-0000-0000-0000-000000000000) is still referenced from table "vm_static".
Where: SQL statement "DELETE FROM vm_static WHERE vm_guid = $1 AND entity_type = 'TEMPLATE'"
PL/pgSQL function "deletevmtemplates" line 7 at SQL statement
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:245) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
Console output rhevm upgrade, no errors seen:
=============================================
[root@istein-31 yum.repos.d]# yum update rhevm-setup
Loaded plugins: product-id, rhnplugin, subscription-manager, versionlock
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
This system is receiving updates from RHN Classic or RHN Satellite.
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package rhevm-setup.noarch 0:3.1.0-43.el6ev will be updated
---> Package rhevm-setup.noarch 0:3.1.0-50.el6ev will be an update
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Updating:
rhevm-setup noarch 3.1.0-50.el6ev upgrade 950 k
Transaction Summary
================================================================================
Upgrade 1 Package(s)
Total download size: 950 k
Is this ok [y/N]: y
Downloading Packages:
rhevm-setup-3.1.0-50.el6ev.noarch.rpm | 950 kB 00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : rhevm-setup-3.1.0-50.el6ev.noarch 1/2
Cleanup : rhevm-setup-3.1.0-43.el6ev.noarch 2/2
Verifying : rhevm-setup-3.1.0-50.el6ev.noarch 1/2
Verifying : rhevm-setup-3.1.0-43.el6ev.noarch 2/2
Updated:
rhevm-setup.noarch 0:3.1.0-50.el6ev
Complete!
[root@istein-31 yum.repos.d]# rhevm-upgrade
Loaded plugins: product-id, rhnplugin
This system is receiving updates from RHN Classic or RHN Satellite.
Checking for updates... (This may take several minutes)
11 Updates available:
* rhevm-3.1.0-50.el6ev.noarch
* rhevm-backend-3.1.0-50.el6ev.noarch
* rhevm-config-3.1.0-50.el6ev.noarch
* rhevm-dbscripts-3.1.0-50.el6ev.noarch
* rhevm-genericapi-3.1.0-50.el6ev.noarch
* rhevm-notification-service-3.1.0-50.el6ev.noarch
* rhevm-restapi-3.1.0-50.el6ev.noarch
* rhevm-tools-common-3.1.0-50.el6ev.noarch
* rhevm-userportal-3.1.0-50.el6ev.noarch
* rhevm-webadmin-portal-3.1.0-50.el6ev.noarch
* vdsm-bootstrap-4.10.2-1.8.el6ev.noarch
Stopping ovirt-engine service... [ DONE ]
Stopping DB related services... [ DONE ]
Cleaning async tasks... [ DONE ]
Info: The following tasks have been found running in the system:
System Tasks:
-----
Task ID: 201
Task name: AddVmTemplateCommand
Task description: Adding a template
[ Apr 24 02:27:19 ] Would you like to proceed and try to stop tasks automatically?
(Answering 'no' will stop the upgrade)? (yes|no): yes
[ Apr 24 02:27:59 ] System will try to clear tasks during the next 3 minutes.
Pre-upgrade validations... [ DONE ]
Backing Up Database... [ DONE ]
Rename Database... [ DONE ]
Updating rpms (This may take a while
depending on your connection speed to the repository.
To track the installation process run tail -f /var/log/yum.log in another session).
Note that the transition from package removal to package installation is expected
to take a significant amount of time. No output will appear in the log file during
this transition.... [ DONE ]
Updating Database... [ DONE ]
Restore Database name... [ DONE ]
Running post install configuration... [ DONE ]
Starting ovirt-engine... [ DONE ]
RHEV Manager upgrade completed successfully!
* Upgrade log available at /var/log/ovirt-engine/ovirt-engine-upgrade_2013_04_24_02_26_34.log
* DB Backup available at /var/lib/ovirt-engine/backups/ovirt-engine_db_backup_2013_04_24_02_26_35.sql
[root@istein-31 yum.repos.d]#
[root@istein-31 yum.repos.d]#
[root@istein-31 yum.repos.d]# rpm -q rhevm
rhevm-3.1.0-50.el6ev.noarch
[root@istein-31 yum.repos.d]#
Created attachment 739442 [details]
engine log
Created attachment 739444 [details]
vdsm log
Created attachment 739446 [details]
Older vdsm log (that include the time in which upgrade stopped async tasks.
Retested for si26.1 (3.1.1) -> si28 (3.14) upgrade: Result same as comment 4. new engine/vdsm logs attached. Cosole log: ========== [root@istein-31 yum.repos.d]# yum update rhevm-setup Loaded plugins: product-id, rhnplugin, subscription-manager, versionlock This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. This system is receiving updates from RHN Classic or RHN Satellite. JBOSS | 1.9 kB 00:00 qa-latest | 1.3 kB 00:00 rhel | 3.9 kB 00:00 rhel-optional | 3.7 kB 00:00 rhevm_deps_acceptance_repo | 1.3 kB 00:00 upgrade | 1.3 kB 00:00 upgrade/primary | 14 kB 00:00 upgrade 65/65 Setting up Update Process Resolving Dependencies --> Running transaction check ---> Package rhevm-setup.noarch 0:3.1.0-43.el6ev will be updated ---> Package rhevm-setup.noarch 0:3.1.0-52.el6ev will be an update --> Finished Dependency Resolution Dependencies Resolved ===================================================================================================================================================================================================== Package Arch Version Repository Size ===================================================================================================================================================================================================== Updating: rhevm-setup noarch 3.1.0-52.el6ev upgrade 952 k Transaction Summary ===================================================================================================================================================================================================== Upgrade 1 Package(s) Total download size: 952 k Is this ok [y/N]: y Downloading Packages: rhevm-setup-3.1.0-52.el6ev.noarch.rpm | 952 kB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Updating : rhevm-setup-3.1.0-52.el6ev.noarch 1/2 Cleanup : rhevm-setup-3.1.0-43.el6ev.noarch 2/2 Verifying : rhevm-setup-3.1.0-52.el6ev.noarch 1/2 Verifying : rhevm-setup-3.1.0-43.el6ev.noarch 2/2 Updated: rhevm-setup.noarch 0:3.1.0-52.el6ev [root@istein-31 yum.repos.d]# rhevm-upgrade Loaded plugins: product-id, rhnplugin This system is receiving updates from RHN Classic or RHN Satellite. Checking for updates... (This may take several minutes) 11 Updates available: * rhevm-3.1.0-52.el6ev.noarch * rhevm-backend-3.1.0-52.el6ev.noarch * rhevm-config-3.1.0-52.el6ev.noarch * rhevm-dbscripts-3.1.0-52.el6ev.noarch * rhevm-genericapi-3.1.0-52.el6ev.noarch * rhevm-notification-service-3.1.0-52.el6ev.noarch * rhevm-restapi-3.1.0-52.el6ev.noarch * rhevm-tools-common-3.1.0-52.el6ev.noarch * rhevm-userportal-3.1.0-52.el6ev.noarch * rhevm-webadmin-portal-3.1.0-52.el6ev.noarch * vdsm-bootstrap-4.10.2-1.10.el6ev.noarch During the upgrade process, RHEV Manager will not be accessible. All existing running virtual machines will continue but you will not be able to start or stop any new virtual machines during the process. Would you like to proceed? (yes|no): yes Stopping ovirt-engine service... [ DONE ] Stopping DB related services... [ DONE ] Cleaning async tasks... [ DONE ] Info: The following tasks have been found running in the system: System Tasks: ----- Task ID: 201 Task name: AddVmTemplateCommand Task description: Adding a template [ Apr 24 23:41:50 ] Would you like to proceed and try to stop tasks automatically? (Answering 'no' will stop the upgrade)? (yes|no): yes [ Apr 24 23:41:59 ] System will try to clear tasks during the next 3 minutes. Pre-upgrade validations... [ DONE ] Backing Up Database... [ DONE ] Rename Database... [ DONE ] Updating rpms (This may take a while depending on your connection speed to the repository. To track the installation process run tail -f /var/log/yum.log in another session). Note that the transition from package removal to package installation is expected to take a significant amount of time. No output will appear in the log file during this transition.... [ DONE ] Updating Database... [ DONE ] Restore Database name... [ DONE ] Running post install configuration... [ DONE ] Starting ovirt-engine... [ DONE ] RHEV Manager upgrade completed successfully! * Upgrade log available at /var/log/ovirt-engine/ovirt-engine-upgrade_2013_04_24_23_39_52.log * DB Backup available at /var/lib/ovirt-engine/backups/ovirt-engine_db_backup_2013_04_24_23_39_54.sql [root@istein-31 yum.repos.d]# rpm -q rhevm rhevm-3.1.0-52.el6ev.noarch Created attachment 739744 [details]
si26.1->si28 engine log
Created attachment 739749 [details]
si26.1->si28 vdsm log
I guess it failedQA because of this exception: 2013-04-24 23:43:00,025 ERROR [org.ovirt.engine.core.bll.EntityAsyncTask] (pool-3-thread-17) EntityAsyncTask::EndCommandAction [within thread]: EndAction for action type AddVmTemplate threw an exception: java.lang.NullPointerException at org.ovirt.engine.core.bll.VmHandler.UnLockVm(VmHandler.java:238) [engine-bll.jar:] at org.ovirt.engine.core.bll.AddVmTemplateCommand.endUnlockOps(AddVmTemplateCommand.java:394) [engine-bll.jar:] at org.ovirt.engine.core.bll.AddVmTemplateCommand.endDefaultOperations(AddVmTemplateCommand.java:384) [engine-bll.jar:] at org.ovirt.engine.core.bll.AddVmTemplateCommand.endSuccessfully(AddVmTemplateCommand.java:370) [engine-bll.jar:] at org.ovirt.engine.core.bll.CommandBase.internalEndSuccessfully(CommandBase.java:512) [engine-bll.jar:] at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:459) [engine-bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1374) [engine-bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:204) [engine-utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:142) [engine-utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:109) [engine-utils.jar:] at org.ovirt.engine.core.bll.CommandBase.endAction(CommandBase.java:405) [engine-bll.jar:] at org.ovirt.engine.core.bll.Backend.endAction(Backend.java:369) [engine-bll.jar:] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_09-icedtea] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_09-icedtea] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_09-icedtea] at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_09-icedtea] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.ovirt.engine.core.utils.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:11) [engine-utils.jar:] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_09-icedtea] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_09-icedtea] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_09-icedtea] at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_09-icedtea] at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:209) [jboss-as-ejb3.jar:7.1.3.Final-redhat-4] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:361) [jboss-as-ejb3.jar:7.1.3.Final-redhat-4] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:192) [jboss-as-ejb3.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:42) [jboss-as-ejb3.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee.jar:7.1.3.Final-redhat-4] at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:181) [jboss-as-ee.jar:7.1.3.Final-redhat-4] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.1.Final-redhat-2] at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee.jar:7.1.3.Final-redhat-4] at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view8.endAction(Unknown Source) [engine-bll.jar:] at org.ovirt.engine.core.bll.EntityAsyncTask.EndCommandAction(EntityAsyncTask.java:135) [engine-bll.jar:] at org.ovirt.engine.core.bll.EntityAsyncTask.access$000(EntityAsyncTask.java:23) [engine-bll.jar:] at org.ovirt.engine.core.bll.EntityAsyncTask$1.run(EntityAsyncTask.java:104) [engine-bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:64) [engine-utils.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_09-icedtea] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_09-icedtea] at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_09-icedtea] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_09-icedtea] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_09-icedtea] at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09-icedtea] Ilanit what was the upgrade/reproduce you did ? Was it according to the steps described in comment #4 ? Why did you restart the engine prior to the upgrade of the engine ? Turning this bug to on QA, as it turns this bug is upgrade fail when there are compensation entries. It will be checked whether there is a need to open a separate bug regaridng the exception in comment 14. Verification flow:
=================
1. Clean install si26.1 (si26.4 do not exist)
2. Insert entry to compensation table, as it is difficult to create a "real" entry in this table.
3. Run upgrade to si28 (3.1.4)
4. Verify no upgrade successful
5. Verify the compensation table was cleared by upgrade.
Console log:
===========
[root@istein-31 yum.repos.d]# psql -U postgres engine
Password for user postgres:
psql (8.4.13)
Type "help" for help.
engine=# select * from business_entity_snapshot ;
id | command_id | command_type | entity_id | entity_type | entity_snapshot | snapshot_class | snapshot_type | insertion_order | started_at
----+------------+--------------+-----------+-------------+-----------------+----------------+---------------+-----------------+------------
(0 rows)
engine=# insert into business_entity_snapshot (id,command_id,command_type) values (uuid_generate_v1(), uuid_generate_v1(), 'aaa');
INSERT 170447 1
engine=# select * from business_entity_snapshot ;
id | command_id | command_type | entity_id | entity_type | entity_snapshot | snapshot_class | snapshot_type | insertion_order | started_at
--------------------------------------+--------------------------------------+--------------+-----------+-------------+-----------------+----------------+---------------+-----------------+-------------------------------
7efec230-af97-11e2-9cec-2f9e4a8784d5 | 7efec3e8-af97-11e2-9c32-efb26df1ff85 | aaa | | | | | | | 2013-04-28 03:06:38.007229+03
(1 row)
engine=# \q
[root@istein-31 yum.repos.d]# vi rhevm.repo
[root@istein-31 yum.repos.d]# yum update rhevm-setup
Loaded plugins: product-id, rhnplugin, subscription-manager, versionlock
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
This system is receiving updates from RHN Classic or RHN Satellite.
JBOSS | 1.9 kB 00:00
qa-latest | 1.3 kB 00:00
rhel | 3.9 kB 00:00
rhel-optional | 3.7 kB 00:00
rhevm_deps_acceptance_repo | 1.3 kB 00:00
upgrade | 1.3 kB 00:00
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package rhevm-setup.noarch 0:3.1.0-43.el6ev will be updated
---> Package rhevm-setup.noarch 0:3.1.0-52.el6ev will be an update
--> Finished Dependency Resolution
Dependencies Resolved
=============================================================================================================================================================================================================================================
Package Arch Version Repository Size
=============================================================================================================================================================================================================================================
Updating:
rhevm-setup noarch 3.1.0-52.el6ev upgrade 952 k
Transaction Summary
=============================================================================================================================================================================================================================================
Upgrade 1 Package(s)
Total download size: 952 k
Is this ok [y/N]: y
Downloading Packages:
rhevm-setup-3.1.0-52.el6ev.noarch.rpm | 952 kB 00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : rhevm-setup-3.1.0-52.el6ev.noarch 1/2
Cleanup : rhevm-setup-3.1.0-43.el6ev.noarch 2/2
Verifying : rhevm-setup-3.1.0-52.el6ev.noarch 1/2
Verifying : rhevm-setup-3.1.0-43.el6ev.noarch 2/2
Updated:
rhevm-setup.noarch 0:3.1.0-52.el6ev
Complete!
[root@istein-31 yum.repos.d]# rhevm-upgrade
Loaded plugins: product-id, rhnplugin
This system is receiving updates from RHN Classic or RHN Satellite.
Checking for updates... (This may take several minutes)
11 Updates available:
* rhevm-3.1.0-52.el6ev.noarch
* rhevm-backend-3.1.0-52.el6ev.noarch
* rhevm-config-3.1.0-52.el6ev.noarch
* rhevm-dbscripts-3.1.0-52.el6ev.noarch
* rhevm-genericapi-3.1.0-52.el6ev.noarch
* rhevm-notification-service-3.1.0-52.el6ev.noarch
* rhevm-restapi-3.1.0-52.el6ev.noarch
* rhevm-tools-common-3.1.0-52.el6ev.noarch
* rhevm-userportal-3.1.0-52.el6ev.noarch
* rhevm-webadmin-portal-3.1.0-52.el6ev.noarch
* vdsm-bootstrap-4.10.2-1.10.el6ev.noarch
During the upgrade process, RHEV Manager will not be accessible.
All existing running virtual machines will continue but you will not be able to
start or stop any new virtual machines during the process.
Would you like to proceed? (yes|no): yes
Stopping ovirt-engine service... [ DONE ]
Stopping DB related services... [ DONE ]
Cleaning async tasks... [ DONE ]
Pre-upgrade validations... [ DONE ]
Backing Up Database... [ DONE ]
Rename Database... [ DONE ]
Updating rpms (This may take a while
depending on your connection speed to the repository.
To track the installation process run tail -f /var/log/yum.log in another session).
Note that the transition from package removal to package installation is expected
to take a significant amount of time. No output will appear in the log file during
this transition.... [ DONE ]
Updating Database... [ DONE ]
Restore Database name... [ DONE ]
Running post install configuration... [ DONE ]
Starting ovirt-engine... [ DONE ]
RHEV Manager upgrade completed successfully!
* Upgrade log available at /var/log/ovirt-engine/ovirt-engine-upgrade_2013_04_28_03_08_10.log
* DB Backup available at /var/lib/ovirt-engine/backups/ovirt-engine_db_backup_2013_04_28_03_08_12.sql
[root@istein-31 yum.repos.d]# rpm -q rhevm
rhevm-3.1.0-52.el6ev.noarch
[root@istein-31 yum.repos.d]# psql -U postgres engine
Password for user postgres:
psql (8.4.13)
Type "help" for help.
engine=# select * from business_entity_snapshot ;
id | command_id | command_type | entity_id | entity_type | entity_snapshot | snapshot_class | snapshot_type | insertion_order | started_at
----+------------+--------------+-----------+-------------+-----------------+----------------+---------------+-----------------+------------
(0 rows)
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-0773.html |