Created attachment 954522 [details] logs from host and engine.log Description of problem: Started a VM in stateless and then attached a disk to it. I powered off the VM and the disk got LOCKED. It appeared that the new disk got deleted. Then I started the VM once again, this time not in stateless. The operation failed with SQLException Version-Release number of selected component (if applicable): rhev 3.5 vt9 rhevm-3.5.0-0.19.beta.el6ev.noarch Host OS: Red Hat Enterprise Linux Server release 7.0 (Maipo) vdsm-4.16.7.3-1.el7ev.x86_64 libvirt-daemon-1.1.1-29.el7_0.3.x86_64 qemu-kvm-rhev-1.5.3-60.el7_0.10.x86_64 selinux-policy-3.13.1-6.el7.noarch sanlock-3.1.0-2.el7.x86_6 How reproducible: Always Steps to Reproduce: 1. Create a VM with disk attached. Start the VM in stateless 2. Attach a new disk to the VM while it is running, hotplug the disk 3. Power-off the VM. The new disk will enter LOCKED because the disk is being deleted 4. Once the disk is removed, start the VM once again, this time not in stateless Actual results: For some reason, starting the VM causes engine to execute snapshot restore for the stateless snapshot. This fails with: 2014-11-06 17:37:19,838 ERROR [org.ovirt.engine.core.bll.RestoreAllSnapshotsCommand] (org.ovirt.thread.pool-7-thread-3) [416028d8] Command org.ovirt.engine.core.bll.RestoreAllSnapshotsCommand throw exception: org. springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000460: Error checking for a transaction at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] Expected results: 1) Not sure why the disk gets removed once the stateless VM is powered-off 2) The stateless snapshot should not be attemted to be restored while starting the VM Additional info: logs from host and engine.log
The stateless flow belongs to the virt team. Moving there for the initial analysis and triaging.
no need to track twice the same thing *** This bug has been marked as a duplicate of bug 1161237 ***
(In reply to Michal Skrivanek from comment #2) > no need to track twice the same thing > > *** This bug has been marked as a duplicate of bug 1161237 *** This is not the same thing. The issue of 1161237 is that the disk remains attached to a stateless VM while it's being removed from the system, which is a different behaviour than a regular disk removal (where the disk is detached from the VM and then removed from the system). The issue here is that when starting the VM AFTER the disk was removed from the system, we get the mentioned exception (which fails the createVM operation)
(In reply to Elad from comment #0) > Expected results: > 1) Not sure why the disk gets removed once the stateless VM is powered-off > 2) The stateless snapshot should not be attemted to be restored while > starting the VM Elad, 1. I see why the disk gets remove: Consider a running stateless VM, it has at least two snapshots: active snapshot and stateless snapshot. The active snapshot is the one which is being used and the stateless snapshot is the one we will switch to when the VM is down. When you attach disk to running stateless VM, and disk is associated with the active snapshot, so when the VM is down and the active snapshot will be removed, the image will be removed as well. 2. The stateless snapshot is attempted to be restored since it could't be restored due to the exception you saw, so when running the VM we try to remove the restore the stateless snapshot. Solving #1 will solve #2. Now about the solution, I'm not sure what is the expected behavior. Well, we definitely don't want to associate attached disk with active snapshot of stateless VM. We can associate them with the stateless snapshot instead. However, in that case, the next time the VM starts we'll take a stateless snapshot of this disk as well and I'm not sure this is the expected result. My question goes like that: you attached disk to running stateless snapshot. What do you expect to happen the next time the VM run? 1. The disk will not be attached to the VM 2. The disk will be attached to the VM but no snapshot will be taken for it 3. The disk will be attached to the VM and stateless snapshot will be taken for it as well IMO, #1 is the best option, what do you think?
*** Bug 1171488 has been marked as a duplicate of this bug. ***
(In reply to Arik from comment #4) > (In reply to Elad from comment #0) > > Expected results: > > 1) Not sure why the disk gets removed once the stateless VM is powered-off > > 2) The stateless snapshot should not be attemted to be restored while > > starting the VM > > Elad, > > 1. I see why the disk gets remove: > Consider a running stateless VM, it has at least two snapshots: active > snapshot and stateless snapshot. The active snapshot is the one which is > being used and the stateless snapshot is the one we will switch to when the > VM is down. > When you attach disk to running stateless VM, and disk is associated with > the active snapshot, so when the VM is down and the active snapshot will be > removed, the image will be removed as well. > > 2. The stateless snapshot is attempted to be restored since it could't be > restored due to the exception you saw, so when running the VM we try to > remove the restore the stateless snapshot. Solving #1 will solve #2. > > Now about the solution, I'm not sure what is the expected behavior. Well, we > definitely don't want to associate attached disk with active snapshot of > stateless VM. We can associate them with the stateless snapshot instead. > However, in that case, the next time the VM starts we'll take a stateless > snapshot of this disk as well and I'm not sure this is the expected result. > > My question goes like that: you attached disk to running stateless snapshot. > What do you expect to happen the next time the VM run? > 1. The disk will not be attached to the VM > 2. The disk will be attached to the VM but no snapshot will be taken for it > 3. The disk will be attached to the VM and stateless snapshot will be taken > for it as well > > IMO, #1 is the best option, what do you think? I agree, the desired behaviour would be that adding a disk to a stateless VM will be blocked. But, if it will be decided that it should be blocked and a disk will be allowed to be plugged to a stateless VM, the disk should not be removed from the setup when the VM stops. It should be detached from the VM and we should inform that this disk will be detached once the VM goes down.
*** Bug 1161237 has been marked as a duplicate of this bug. ***
Verified with rhevm-3.6.0.3-0.1.el6.noarch. 1. Created stateless VM with disk_A 2. started VM (snapshot is created for restoration) 3. Created disk_B 4. Attached disk_B to running stateless VM 5. Stopped VM. result: disk_B is detached from stateless VM. VM is down. Great success.