Created attachment 941720 [details] engine log Description of problem: When creating snapshot including memory snapshot, a new volume is created for the memory contents. When removing a vm, the memory snapshot is not deleted. Version-Release number of selected component (if applicable): oVirt Engine Version: 3.5.0-0.0.master.20140911091402.gite1c5ffd.fc20 vdsm master 29defc3faab3d7 How reproducible: Always Steps to Reproduce: 1. Create vm with 1G memory and one 1G thin provisioning disk 2. Run vm 3. Create snapshot (ensure that "save memory" is checked) 4. Stop vm 5. Remove vm Actual results: Memory volumes not removed, and the domain holding them free memory is smaller then expected (10G instead of 15G). The only way to remove the volume is to login to one of the hosts and remove the unwanted lvs, but there is no easy way to detect that these volumes are indeed the unused. Expected results: Memory volumes should be removed when vm is removed Additional info: For testing this issue, I created two new iscsi storage domains, and repeated the steps above 2 times. This is the master domain contents at the end of the test. # lvs ff559f46-c495-4f6b-901c-2a624042a050 LV VG Attr LSize 3149a0f8-82d3-43a4-b7e6-d6033485afb0 ff559f46-c495-4f6b-901c-2a624042a050 -wi------- 128.00m 75b7d7b4-5c1a-48fe-a079-0f6529bd8968 ff559f46-c495-4f6b-901c-2a624042a050 -wi------- 128.00m ids ff559f46-c495-4f6b-901c-2a624042a050 -wi-ao---- 128.00m inbox ff559f46-c495-4f6b-901c-2a624042a050 -wi-a----- 128.00m leases ff559f46-c495-4f6b-901c-2a624042a050 -wi-a----- 2.00g master ff559f46-c495-4f6b-901c-2a624042a050 -wi-ao---- 1.00g metadata ff559f46-c495-4f6b-901c-2a624042a050 -wi-a----- 512.00m outbox ff559f46-c495-4f6b-901c-2a624042a050 -wi-a----- 128.00m The two 128M volumes are ovf store volumes. This is the other domain contents at the end of the test: # lvs 3e419414-ee73-47e8-809d-60de4a88403c LV VG Attr LSize 0ffd2268-ef48-4fbf-aaf6-73f4249d5f40 3e419414-ee73-47e8-809d-60de4a88403c -wi------- 1.38g 58b9b9f8-b2c7-441f-acdb-2c0616dff271 3e419414-ee73-47e8-809d-60de4a88403c -wi------- 1.38g 70aafeeb-6307-4451-a558-d707310667ab 3e419414-ee73-47e8-809d-60de4a88403c -wi------- 1.00g e89a3dc7-947c-4692-b5d1-eb8f5a4958cb 3e419414-ee73-47e8-809d-60de4a88403c -wi------- 1.00g ids 3e419414-ee73-47e8-809d-60de4a88403c -wi-ao---- 128.00m inbox 3e419414-ee73-47e8-809d-60de4a88403c -wi-a----- 128.00m leases 3e419414-ee73-47e8-809d-60de4a88403c -wi-a----- 2.00g master 3e419414-ee73-47e8-809d-60de4a88403c -wi-a----- 1.00g metadata 3e419414-ee73-47e8-809d-60de4a88403c -wi-a----- 512.00m outbox 3e419414-ee73-47e8-809d-60de4a88403c -wi-a----- 128.00m Note that the vm was created on the master domain (ff559f46-c495-4f6b-901c-2a624042a050), but the memory volumes are created on the other domain (3e419414-ee73-47e8-809d-60de4a88403c) - this does not make sense and probably the root cause of this. Looking in vdsm log, we can see that only images in the master domain (ff559f46-c495-4f6b-901c-2a624042a050) were deleted. There is no other deleteImage or deleteVolume request in vdsm log. # grep 'Run and protect: deleteImage' vdsm.log Thread-34::INFO::2014-09-26 22:57:25,005::logUtils::48::dispatcher::(wrapper) Run and protect: deleteImage(sdUUID='ff559f46-c495-4f6b-901c-2a624042a050', spUUID='b86b687a-d073-497a-ac8a-249025419a3e', imgUUID='a256fc11-f9ab-4391-af78-2a96e27fe39b', postZero='false', force='false') Thread-34::INFO::2014-09-26 22:57:25,462::logUtils::51::dispatcher::(wrapper) Run and protect: deleteImage, Return response: None Thread-34::INFO::2014-09-26 23:00:45,883::logUtils::48::dispatcher::(wrapper) Run and protect: deleteImage(sdUUID='ff559f46-c495-4f6b-901c-2a624042a050', spUUID='b86b687a-d073-497a-ac8a-249025419a3e', imgUUID='9288c7e8-43fd-4174-9889-634fb2e0437a', postZero='false', force='false') Thread-34::INFO::2014-09-26 23:00:46,338::logUtils::51::dispatcher::(wrapper) Run and protect: deleteImage, Return response: None Thread-34::INFO::2014-09-26 23:07:24,205::logUtils::48::dispatcher::(wrapper) Run and protect: deleteImage(sdUUID='ff559f46-c495-4f6b-901c-2a624042a050', spUUID='b86b687a-d073-497a-ac8a-249025419a3e', imgUUID='60e28edc-4bb3-4f87-b6a2-5dc45f1a0971', postZero='false', force='false') Thread-34::INFO::2014-09-26 23:07:24,659::logUtils::51::dispatcher::(wrapper) Run and protect: deleteImage, Return response: None Since vdsm was not asked to delete the images, this is clearly an engine bug. I did not test it with older engine version, but I'm sure this is a regression. I'm creating and deleting vms regularly while verifying, and I would notice if my storage domain loose free space.
Created attachment 941721 [details] vdsm log
Idan, I think this is another instance of a bug you're already working on. Please verify (and solve :-))
oVirt 3.5 has been released and should include the fix for this issue.