Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 982971

Summary:	VM disk corruption when a user deletes one of the snapshots.
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	Nir Magnezi <nmagnezi>
Component:	ovirt-engine	Assignee:	Nobody's working on this, feel free to take it <nobody>
Status:	CLOSED DUPLICATE	QA Contact:
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	3.2.0	CC:	acathrow, amureini, fsimonce, iheim, jkt, lpeer, Rhev-m-bugs, scohen, yeylon
Target Milestone:	---	Keywords:	Triaged
Target Release:	3.2.2
Hardware:	Unspecified
OS:	Linux
Whiteboard:	storage
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-07-25 08:07:36 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Storage	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Nir Magnezi 2013-07-10 08:53:00 UTC

Description of problem:
=======================
I Deleted one of my VM snapshot and by that, Destroyed it's OS.
I had two snapshots for this VM: snapshot #1 with a clean OS and snapshot #2 with OS and some more data.
Since I no longer needed the first snapshot, I deleted it.

I'd expect some warning with:
1. Explanation that there is a dependency between snapshots.
2. All other snapshots "based" on the snapshot I'm about to delete will become unusable.

I didn't get any warning or anything else that could give me the impression I'm doing something wronge and I'm about to lose my data.

Version-Release number of selected component (if applicable):
=============================================================
3.2.z

How reproducible:
=================
2/2

Steps to Reproduce:
===================
1. Created a VM
2. Installed OS
3. Took a snapshot #1 (while vm is powered on)
4. Created a test txt file.
5. Took a snapshot #2 (while vm is powered on)
6. VM Power off
7. Preview of snapshot 2
8. Commit of snapshot 2
9. VM power on
10. Verified that the Operating System is running
11. VM power off
12. Deleted snapshot 1
13. VM power on

Actual results:
===============
Due to the disk corruption, The OS won't boot
The error I get is: No bootable device.

Expected results:
=================
Either a warning as mentioned in 'Description of problem' or a solution to prevent this from happening.

Comment 1 Federico Simoncelli 2013-07-15 11:59:03 UTC

VDSM version:

vdsm-4.10.2-19.0.el6ev.x86_64

History of the image:

Thread-672185::INFO::2013-07-09 15:06:22,745::logUtils::40::dispatcher::(wrapper) Run and protect: createVolume(sdUUID='eaf45b65-9c2d-4c57-ab07-4366ff58ef3f', spUUID='e3f12c2e-9742-4b83-8c6f-dc397df2017b', imgUUID='bc94572a-32b6-478e-a59a-5282465b7f25', size='85899345920', volFormat=5, preallocate=1, diskType=2, volUUID='158f4c06-a36d-455c-a506-9dddd1cde4f2', desc='', srcImgUUID='00000000-0000-0000-0000-000000000000', srcVolUUID='00000000-0000-0000-0000-000000000000')
Thread-673046::INFO::2013-07-09 15:30:22,844::logUtils::40::dispatcher::(wrapper) Run and protect: createVolume(sdUUID='eaf45b65-9c2d-4c57-ab07-4366ff58ef3f', spUUID='e3f12c2e-9742-4b83-8c6f-dc397df2017b', imgUUID='bc94572a-32b6-478e-a59a-5282465b7f25', size='85899345920', volFormat=4, preallocate=2, diskType=2, volUUID='7cb6d029-627d-4da5-89f4-651d025f5a22', desc='', srcImgUUID='bc94572a-32b6-478e-a59a-5282465b7f25', srcVolUUID='158f4c06-a36d-455c-a506-9dddd1cde4f2')
Thread-673541::INFO::2013-07-09 15:43:58,173::logUtils::40::dispatcher::(wrapper) Run and protect: createVolume(sdUUID='eaf45b65-9c2d-4c57-ab07-4366ff58ef3f', spUUID='e3f12c2e-9742-4b83-8c6f-dc397df2017b', imgUUID='bc94572a-32b6-478e-a59a-5282465b7f25', size='85899345920', volFormat=4, preallocate=2, diskType=2, volUUID='b0f74f82-8a9b-4a56-b19f-4a6a9da842de', desc='', srcImgUUID='bc94572a-32b6-478e-a59a-5282465b7f25', srcVolUUID='158f4c06-a36d-455c-a506-9dddd1cde4f2')
Thread-673920::INFO::2013-07-09 15:54:26,948::logUtils::40::dispatcher::(wrapper) Run and protect: deleteVolume(sdUUID='eaf45b65-9c2d-4c57-ab07-4366ff58ef3f', spUUID='e3f12c2e-9742-4b83-8c6f-dc397df2017b', imgUUID='bc94572a-32b6-478e-a59a-5282465b7f25', volumes=['7cb6d029-627d-4da5-89f4-651d025f5a22'], postZero='false', force='true')
Thread-674716::INFO::2013-07-09 16:16:51,946::logUtils::40::dispatcher::(wrapper) Run and protect: createVolume(sdUUID='eaf45b65-9c2d-4c57-ab07-4366ff58ef3f', spUUID='e3f12c2e-9742-4b83-8c6f-dc397df2017b', imgUUID='bc94572a-32b6-478e-a59a-5282465b7f25', size='85899345920', volFormat=4, preallocate=2, diskType=2, volUUID='8f9aa50c-cee3-4c63-a4d6-c9080d6b993c', desc='', srcImgUUID='bc94572a-32b6-478e-a59a-5282465b7f25', srcVolUUID='b0f74f82-8a9b-4a56-b19f-4a6a9da842de')
Thread-674939::INFO::2013-07-09 16:22:53,685::logUtils::40::dispatcher::(wrapper) Run and protect: createVolume(sdUUID='eaf45b65-9c2d-4c57-ab07-4366ff58ef3f', spUUID='e3f12c2e-9742-4b83-8c6f-dc397df2017b', imgUUID='bc94572a-32b6-478e-a59a-5282465b7f25', size='85899345920', volFormat=4, preallocate=2, diskType=2, volUUID='c7531323-8212-4c5b-95c9-bb7f71a41616', desc='', srcImgUUID='bc94572a-32b6-478e-a59a-5282465b7f25', srcVolUUID='b0f74f82-8a9b-4a56-b19f-4a6a9da842de')
Thread-674974::INFO::2013-07-09 16:23:43,430::logUtils::40::dispatcher::(wrapper) Run and protect: deleteVolume(sdUUID='eaf45b65-9c2d-4c57-ab07-4366ff58ef3f', spUUID='e3f12c2e-9742-4b83-8c6f-dc397df2017b', imgUUID='bc94572a-32b6-478e-a59a-5282465b7f25', volumes=['8f9aa50c-cee3-4c63-a4d6-c9080d6b993c'], postZero='false', force='true')


158f4c06 <= 7cb6d029 (deleted)
         \= b0f74f82 <= 8f9aa50c (deleted)
                     \= c7531323

158f4c06 <= b0f74f82 <= c7531323


Merge command:

Thread-675730::INFO::2013-07-09 16:44:51,180::logUtils::40::dispatcher::(wrapper) Run and protect: mergeSnapshots(sdUUID='eaf45b65-9c2d-4c57-ab07-4366ff58ef3f', spUUID='e3f12c2e-9742-4b83-8c6f-dc397df2017b', vmUUID='', imgUUID='bc94572a-32b6-478e-a59a-5282465b7f25', ancestor='158f4c06-a36d-455c-a506-9dddd1cde4f2', successor='b0f74f82-8a9b-4a56-b19f-4a6a9da842de', postZero='false')
0116f3de-142e-452e-96d0-6fa429df6a92::INFO::2013-07-09 16:44:53,367::image::1162::Storage.Image::(merge) merge with convert: src = /rhev/data-center/e3f12c2e-9742-4b83-8c6f-dc397df2017b/eaf45b65-9c2d-4c57-ab07-4366ff58ef3f/images/bc94572a-32b6-478e-a59a-5282465b7f25/b0f74f82-8a9b-4a56-b19f-4a6a9da842de dst = /rhev/data-center/e3f12c2e-9742-4b83-8c6f-dc397df2017b/eaf45b65-9c2d-4c57-ab07-4366ff58ef3f/images/bc94572a-32b6-478e-a59a-5282465b7f25/158f4c06-a36d-455c-a506-9dddd1cde4f2


b0f74f82 <= c7531323


Debugging current state, everything looks in order:

b0f74f82-8a9b-4a56-b19f-4a6a9da842de eaf45b65-9c2d-4c57-ab07-4366ff58ef3f 80.00g  IU_bc94572a-32b6-478e-a59a-5282465b7f25,PU_00000000-0000-0000-0000-000000000000,MD_121
c7531323-8212-4c5b-95c9-bb7f71a41616 eaf45b65-9c2d-4c57-ab07-4366ff58ef3f  1.00g  IU_bc94572a-32b6-478e-a59a-5282465b7f25,PU_b0f74f82-8a9b-4a56-b19f-4a6a9da842de,MD_120

# dmsetup table | grep c7531323--8212--4c5b--95c9--bb7f71a41616
eaf45b65--9c2d--4c57--ab07--4366ff58ef3f-c7531323--8212--4c5b--95c9--bb7f71a41616: 0 2097152 linear 253:2 1786775552
# dmsetup table | grep b0f74f82--8a9b--4a56--b19f--4a6a9da842de
eaf45b65--9c2d--4c57--ab07--4366ff58ef3f-b0f74f82--8a9b--4a56--b19f--4a6a9da842de: 0 167772160 linear 253:2 2254440448

# qemu-img info /dev/eaf45b65-9c2d-4c57-ab07-4366ff58ef3f/b0f74f82-8a9b-4a56-b19f-4a6a9da842de
image: /dev/eaf45b65-9c2d-4c57-ab07-4366ff58ef3f/b0f74f82-8a9b-4a56-b19f-4a6a9da842de
file format: raw
virtual size: 80G (85899345920 bytes)
disk size: 0

# qemu-img info /dev/eaf45b65-9c2d-4c57-ab07-4366ff58ef3f/c7531323-8212-4c5b-95c9-bb7f71a41616
image: /dev/eaf45b65-9c2d-4c57-ab07-4366ff58ef3f/c7531323-8212-4c5b-95c9-bb7f71a41616
file format: qcow2
virtual size: 80G (85899345920 bytes)
disk size: 0
cluster_size: 65536
backing file: ../bc94572a-32b6-478e-a59a-5282465b7f25/b0f74f82-8a9b-4a56-b19f-4a6a9da842de (actual path: /dev/eaf45b65-9c2d-4c57-ab07-4366ff58ef3f/../bc94572a-32b6-478e-a59a-5282465b7f25/b0f74f82-8a9b-4a56-b19f-4a6a9da842de)


The problem was related to the HSM having the old LVs still active with the old/wrong mapping:

# dmsetup table | grep c7531323
eaf45b65--9c2d--4c57--ab07--4366ff58ef3f-c7531323--8212--4c5b--95c9--bb7f71a41616: 0 2097152 linear 253:2 1786775552

# dmsetup table | grep b0f74f82
eaf45b65--9c2d--4c57--ab07--4366ff58ef3f-b0f74f82--8a9b--4a56--b19f--4a6a9da842de: 0 2097152 linear 253:2 1859651584


Note: using "lvchange -an" to deactivate the LVs was not enough (probably because of the mismatch between the maps), so I had to manually remove the dm:

# dmsetup remove eaf45b65--9c2d--4c57--ab07--4366ff58ef3f-b0f74f82--8a9b--4a56--b19f--4a6a9da842de

After that the VM was able to boot properly.

This is a duplicate of bug 962549, I suggest to upgrade to (at least) vdsm-4.10.2-22.0.el6ev

Comment 2 Federico Simoncelli 2013-07-15 12:01:39 UTC

Created attachment 773705 [details]
vdsm-logs.tar.gz

Comment 3 Allon Mureinik 2013-07-25 08:07:36 UTC

Closing as duplicate, as per comment #1.

*** This bug has been marked as a duplicate of bug 962549 ***