Bug 971022 - engine: LiveMigrateDisk: java.lang.reflect.InvocationTargetException when vm is removed before the deleteImage step is cleared on wipe=true disk
Summary: engine: LiveMigrateDisk: java.lang.reflect.InvocationTargetException when vm...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 3.2.0
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: 3.4.0
Assignee: Daniel Erez
QA Contact: Leonid Natapov
URL:
Whiteboard: storage
Depends On:
Blocks: rhev3.4beta 1142926
TreeView+ depends on / blocked
 
Reported: 2013-06-05 13:27 UTC by Dafna Ron
Modified: 2016-02-10 20:34 UTC (History)
10 users (show)

Fixed In Version: ovirt-3.4.0-alpha1
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:
oVirt Team: Storage
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
logs (3.81 MB, application/x-gzip)
2013-06-05 13:27 UTC, Dafna Ron
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 22882 0 None None None Never

Description Dafna Ron 2013-06-05 13:27:31 UTC
Created attachment 757186 [details]
logs

Description of problem:

due to bug 970969 in which the deleteImage step continues to run after the lock on the disk is freed, I did a test in which I stopped the vm once we start the deleteImage step in LSM and than removed the vm. 
the disk of the vm is deleted but we fail to clear the deleteImage task for the LSM command and log shows the following exception in a loop: 

2013-06-05 16:12:57,831 ERROR [org.ovirt.engine.core.bll.CommandsFactory] (pool-4-thread-48) [5748dd98] CommandsFactory [parameter: VdcActionParametersBase]: Failed to get type
 information using reflection for Action: LiveMigrateDisk: java.lang.reflect.InvocationTargetException


Version-Release number of selected component (if applicable):

sf14

How reproducible:

100%

Steps to Reproduce:
1. in iscsi storage with two hosts create and run a vm with 1 preallocated wipe=true disk
2. run the vm
3. LSM the vm's disk
4. when the deleteImage step starts and the vm's disk is no longer locked power off the vm
5. remove the vm

Actual results:

the vm and its disk is removed but engine fails to clear the LSM task and we run it in a loop with exception. 

Expected results:

if freeing the disk lock during deleteImage is run, we should handle the exception and clear the old task. 

Additional info: logs


06b87b26-ef82-4fc9-b2dd-02f8de2cb9e3 :
        verb = deleteImage
        id = 06b87b26-ef82-4fc9-b2dd-02f8de2cb9e3


'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {'06b87b26-ef82-4fc9-b2dd-02f8de2cb9e3': {'message': '1 jobs completed successfully', 'code': 0, 'taskID': '06b87b26-
ef82-4fc9-b2dd-02f8de2cb9e3', 'taskResult': 'success', 'taskState': 'finished'}}}



LiveMigrateDisk, executionIndex: 2
2013-06-05 16:12:57,831 ERROR [org.ovirt.engine.core.bll.CommandsFactory] (pool-4-thread-48) [5748dd98] CommandsFactory [parameter: VdcActionParametersBase]: Failed to get type
 information using reflection for Action: LiveMigrateDisk: java.lang.reflect.InvocationTargetException
        at sun.reflect.GeneratedConstructorAccessor121.newInstance(Unknown Source) [:1.7.0_19]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.7.0_19]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525) [rt.jar:1.7.0_19]
        at org.ovirt.engine.core.bll.CommandsFactory.CreateCommand(CommandsFactory.java:45) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.Backend.endAction(Backend.java:377) [engine-bll.jar:]
        at sun.reflect.GeneratedMethodAccessor228.invoke(Unknown Source) [:1.7.0_19]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_19]
        at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_19]
        at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:
72) [jboss-as-ee.jar:7.2.0.Final-redhat-8]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2]
        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.1.Final-redhat-2]
        at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.0.Final-redhat-8]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2]
        at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.1.Final-redhat-2]
        at org.ovirt.engine.core.utils.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:11) [engine-utils.jar:]
        at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) [:1.7.0_19]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_19]
        at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_19]
        at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee.jar:7.2.0.Final-redhat-8]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2]
        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.1.Final-redhat-2]
        at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee.jar:7.2.0.Final-redhat-8]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2]
        at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.2.0.Final-redhat-8]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2]
        at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.1.Final-redhat-2]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-2]
        at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.1.Final-redhat-2]
        at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.2.0.Final-redhat-8]

Comment 3 Sandro Bonazzola 2014-01-14 08:44:48 UTC
ovirt 3.4.0 alpha has been released

Comment 4 Leonid Natapov 2014-02-17 13:48:39 UTC
3.4.0-0.7.beta2.el6. fixed. lsm task is cleared.

Comment 5 Itamar Heim 2014-06-12 14:11:18 UTC
Closing as part of 3.4.0


Note You need to log in before you can comment on or make changes to this bug.