Bug 828192 - ovirt-engine-backend: remove disk will not remove the disk from the snapshot after merge action on multiple disk vm resulting with NPE when trying to preview snapshot
Summary: ovirt-engine-backend: remove disk will not remove the disk from the snapshot ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 3.1.0
Hardware: x86_64
OS: Linux
medium
high
Target Milestone: ---
: 3.1.0
Assignee: Liron Aravot
QA Contact: Dafna Ron
URL:
Whiteboard: storage
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-06-04 12:39 UTC by Dafna Ron
Modified: 2016-02-10 17:04 UTC (History)
11 users (show)

Fixed In Version: SI17
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-12-04 20:06:03 UTC
oVirt Team: Storage
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
logs (654.97 KB, application/x-gzip)
2012-06-04 12:42 UTC, Dafna Ron
no flags Details
log and screen shot (224.55 KB, application/x-gzip)
2012-07-30 13:57 UTC, Dafna Ron
no flags Details
log (165.76 KB, application/x-xz)
2012-07-30 14:10 UTC, Dafna Ron
no flags Details

Description Dafna Ron 2012-06-04 12:39:49 UTC
Description of problem:

I deleted a snapshot on a vm with multiple disks (which caused a collaps action on teh chain) and then removed one of the disks. 
the disk I removed remains in the snapshot which causes a NPE and failure when trying to preview the snapshot



Version-Release number of selected component (if applicable):

si4

How reproducible:

100%

Steps to Reproduce:
1. create a vm with several disks
2. create 2 snapshots
3. delete the first snapshot (merge)
4. remove one of the disks

Actual results:

if we expend the snapshot to see the images we can see that we still have 3 disks in the snapshot -> if we try to preview the snapshot we will fail with NPE

Expected results:

we should remove the disk image from the snapshot

Additional info: vdsm and backend logs

2012-06-01 15:31:56,088 ERROR [org.ovirt.engine.core.bll.EntityAsyncTask] (pool-10-thread-43) EntityAsyncTask::EndCommandAction [within thread]: EndAction for action type TryBackToAllSnapshotsOfVm threw an exception: java.lang.NullPointe
rException
        at org.ovirt.engine.core.bll.ImagesHandler.getAllImageSnapshots(ImagesHandler.java:332) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.VmCommand.UpdateVmInSpm(VmCommand.java:198) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.VmCommand.UpdateVmInSpm(VmCommand.java:179) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.TryBackToAllSnapshotsOfVmCommand.EndSuccessfully(TryBackToAllSnapshotsOfVmCommand.java:76) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.InternalEndSuccessfully(CommandBase.java:465) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:420) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1157) [engine-bll.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:204) [engine-utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:142) [engine-utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:109) [engine-utils.jar:]
        at org.ovirt.engine.core.bll.CommandBase.EndAction(CommandBase.java:366) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.Backend.endAction(Backend.java:355) [engine-bll.jar:]
        at sun.reflect.GeneratedMethodAccessor360.invoke(Unknown Source) [:1.6.0_24]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.6.0_24]
        at java.lang.reflect.Method.invoke(Method.java:616) [rt.jar:1.6.0_24]
        at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.1.1.Final-redhat-1]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-1]
        at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.1.Final-redhat-1]
        at org.ovirt.engine.core.utils.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:11) [engine-utils.jar:]
        at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) [:1.6.0_24]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.6.0_24]
        at java.lang.reflect.Method.invoke(Method.java:616) [rt.jar:1.6.0_24]
        at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee.jar:7.1.1.Final-
redhat-1]

Comment 1 Dafna Ron 2012-06-04 12:42:10 UTC
Created attachment 589136 [details]
logs

Comment 2 Dafna Ron 2012-06-04 12:50:17 UTC
after preview fails we also cannot remove vm/disks/snapshots because snapshot remains in image locked.

Comment 4 Allon Mureinik 2012-06-19 09:30:01 UTC
Agree the test fails, but this should not be a test blocker - please specify what flows it blocks or remove TestBlocker comment.

Also, we need to discuss the correct behavior here.
Removing a disk across all the snapshots does not sound like the behavior we'd want.

Comment 8 Allon Mureinik 2012-07-02 14:28:37 UTC
patch uploaded to gerrit: http://gerrit.ovirt.org/#/c/5831/

Changing the overall behavior of removing a disk may be too difficult/risky at this stage, but that's no reason you shouldn't be able to run a VM :-)

Comment 11 Allon Mureinik 2012-07-03 13:09:55 UTC
reopening - after further review, this is not how we want to handle this issue.

Comment 13 Allon Mureinik 2012-07-08 18:22:06 UTC
After discussions, we decided to remove the disk from all the snapshots, instead of leaving illegals.

patch in gerrit: http://gerrit.ovirt.org/#/c/6025/

Comment 14 Allon Mureinik 2012-07-23 16:05:55 UTC
Patch was merged.

Steps to Reproduce:
1. create a vm with several disks
2. create 2 snapshots
3. delete the first snapshot (merge)
4. remove one of the disks

You will see the disk /removed/ from the snapshot.

Comment 15 Dafna Ron 2012-07-30 13:56:18 UTC
disks were removed from snapshot only when located on the same domain. 
when I created a vm with disks located on different domains one of the disks in snapshot became illegal but was not removed 

snapshot remains in image locked. 

log attached + screen shot


reproduce: 

1. create a vm with several disks located on two different SD
2. create 2 snapshots
3. delete the first snapshot
4. remove disks from both domains

Comment 16 Dafna Ron 2012-07-30 13:57:34 UTC
Created attachment 601280 [details]
log and screen shot

Comment 17 Dafna Ron 2012-07-30 14:10:23 UTC
Created attachment 601285 [details]
log

operation finally ended with failure with same exception: 

2012-07-30 17:13:42,095 ERROR [org.ovirt.engine.core.bll.EntityAsyncTask] (pool-4-thread-41) EntityAsyncTask::EndCommandAction [within thread]: EndAction for action type TryBackToAllSnapshotsOfVm threw an exception: java.lang.NullPointe
rException
        at org.ovirt.engine.core.bll.ImagesHandler.getAllImageSnapshots(ImagesHandler.java:344) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.VmCommand.UpdateVmInSpm(VmCommand.java:182) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.VmCommand.UpdateVmInSpm(VmCommand.java:164) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.TryBackToAllSnapshotsOfVmCommand.restoreVmConfigFromSnapshot(TryBackToAllSnapshotsOfVmCommand.java:96) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.TryBackToAllSnapshotsOfVmCommand.EndSuccessfully(TryBackToAllSnapshotsOfVmCommand.java:76) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.InternalEndSuccessfully(CommandBase.java:476) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:431) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1243) [engine-bll.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:204) [engine-utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:142) [engine-utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:109) [engine-utils.jar:]
        at org.ovirt.engine.core.bll.CommandBase.EndAction(CommandBase.java:377) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.Backend.endAction(Backend.java:355) [engine-bll.jar:]
        at sun.reflect.GeneratedMethodAccessor423.invoke(Unknown Source) [:1.7.0_05-icedtea]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_05-icedtea]
        at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_05-icedtea]
        at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee.jar:7.1.2.Final-redhat-1]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final-redhat-1]
        at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.1.Final-redhat-1]
        at org.ovirt.engine.core.utils.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:11) [engine-utils.jar:]
        at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_05-icedtea]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_05-icedtea]
        at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_05-icedtea]
        at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee.jar:7.1.2.Final
-redhat-1]


new log attached

Comment 18 Allon Mureinik 2012-08-01 08:50:26 UTC
Tested with commit hash cb6eea48bc31cd63caeb840eaec6f0abf133df93, was unable to recreate the issue.

Need QE's help to reproduce, or at least undestand how my reproduction is wrong.

Comment 19 Dafna Ron 2012-08-01 09:14:20 UTC
I wrote the reproduction in comment 15. 
we are working with si12 and vdsm-4.9.6-24.0.el6_3.x86_64

Comment 20 Haim 2012-08-03 13:25:32 UTC
(In reply to comment #19)
> I wrote the reproduction in comment 15. 
> we are working with si12 and vdsm-4.9.6-24.0.el6_3.x86_64

moving back to assigned per comment 19 - there was no fix provided by R&D and bug was failed QA.
please resolve.

Comment 21 Allon Mureinik 2012-08-04 16:29:43 UTC
Error was reproduced.
I believe the problem reproducing (see comment #18) was that I did not attempt to remove the disks simoltaniously, thus not encountering the race condition in snapshot updating.

Working on a solution.

Comment 24 Liron Aravot 2012-09-02 14:21:06 UTC
submitted the following patch as fix : http://gerrit.ovirt.org/#/c/7482/

Comment 25 Allon Mureinik 2012-09-03 07:28:42 UTC
merged change id Iccb44f1aa9d204477955343167133849a4146753

Comment 26 Dafna Ron 2012-09-09 10:48:40 UTC
verified on si17


Note You need to log in before you can comment on or make changes to this bug.