Bug 738714

Summary:	splitting off final mirror image doesn't always work
Product:	Red Hat Enterprise Linux 6	Reporter:	Corey Marthaler <cmarthal>
Component:	lvm2	Assignee:	Jonathan Earl Brassow <jbrassow>
Status:	CLOSED ERRATA	QA Contact:	Corey Marthaler <cmarthal>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	6.2	CC:	agk, dwysocha, heinzm, jbrassow, mbroz, prajnoha, prockai, thornber, zkabelac
Target Milestone:	rc	Keywords:	Regression, TestBlocker
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	lvm2-2.02.87-3.el6	Doc Type:	Bug Fix
Doc Text:	Do not document.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2011-12-06 17:03:15 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	743047

Description Corey Marthaler 2011-09-15 15:52:13 UTC

Description of problem:
This bug originates from bug 733114, but I fear that 733114 is becoming just about the I/O errors that can occur with this operation and not about the fact the the operation still doesn't even work with the latest rpms. It seems that the more legs the mirror has, the greater the odds that split attempt will fail.

[root@taft-01 ~]# lvs -a -o +devices
 LV                         Attr   LSize   Log                  Copy%  Devices
 split_same_name            mwi-a- 300.00m split_same_name_mlog 100.00 split_same_name_mimage_0(0),split_same_name_mimage_1(0),split_same_name_mimage_2(0)
 [split_same_name_mimage_0] iwi-ao 300.00m                             /dev/sdh1(0)
 [split_same_name_mimage_1] iwi-ao 300.00m                             /dev/sdg1(0)
 [split_same_name_mimage_2] iwi-ao 300.00m                             /dev/sdf1(0)
 [split_same_name_mlog]     lwi-ao   4.00m                             /dev/sdc1(0)

[root@taft-01 ~]# lvconvert --splitmirrors 1 --name new split_image/split_same_name
  LV split_image/split_same_name_mimage_2 in use: not deactivating
  Failed to deactivate former mirror image, split_same_name_mimage_2

[root@taft-01 ~]# lvs -a -o +devices
 LV                         Attr   LSize   Log                  Copy%  Devices                                                
 split_same_name            mwi-a- 300.00m split_same_name_mlog 100.00 split_same_name_mimage_0(0),split_same_name_mimage_1(0)
 [split_same_name_mimage_0] iwi-ao 300.00m                             /dev/sdh1(0)
 [split_same_name_mimage_1] iwi-ao 300.00m                             /dev/sdg1(0)
 split_same_name_mimage_2   -wi-a- 300.00m                             /dev/sdf1(0)
 [split_same_name_mlog]     lwi-ao   4.00m                             /dev/sdc1(0)

Version-Release number of selected component (if applicable):
[root@taft-01 ~]# /usr/tests/sts-rhel6.2/lvm2/bin/lvm_rpms 
2.6.32-195.el6.x86_64

lvm2-2.02.87-2.1.el6    BUILT: Wed Sep 14 09:44:16 CDT 2011
lvm2-libs-2.02.87-2.1.el6    BUILT: Wed Sep 14 09:44:16 CDT 2011
lvm2-cluster-2.02.87-2.1.el6    BUILT: Wed Sep 14 09:44:16 CDT 2011
udev-147-2.38.el6    BUILT: Fri Sep  9 16:25:50 CDT 2011
device-mapper-1.02.66-2.1.el6    BUILT: Wed Sep 14 09:44:16 CDT 2011
device-mapper-libs-1.02.66-2.1.el6    BUILT: Wed Sep 14 09:44:16 CDT 2011
device-mapper-event-1.02.66-2.1.el6    BUILT: Wed Sep 14 09:44:16 CDT 2011
device-mapper-event-libs-1.02.66-2.1.el6    BUILT: Wed Sep 14 09:44:16 CDT 2011
cmirror-2.02.87-2.1.el6    BUILT: Wed Sep 14 09:44:16 CDT 2011


How reproducible:
Most of the time

Comment 1 Jonathan Earl Brassow 2011-09-16 16:45:03 UTC

After the suspend/resume cycle following the split, sync_local_dev_names() must be called to settle udev before calling deactivate_lv.  This avoids udev opening the device and colliding with the removal.

Fix passes 1000 iterations of 4-way mirror creation followed by 3 sequential single leg split-offs.

Checked-in upstream in Version 2.02.89

Comment 2 Jonathan Earl Brassow 2011-09-16 18:41:26 UTC

git commit id:
5c11544dc10212f5839dde0c97ec86be32c2b4c5

Comment 3 Corey Marthaler 2011-09-19 20:48:22 UTC

This issue appears to apply to a broader list of lvconvert operations then just --splitmirrors. Mirror leg and log conversion also appears to be broken, which affects device failure and conversion testing. Hopefully the fix for this mentioned in comment #1 affects the other convert options as well.

Comment 5 Corey Marthaler 2011-09-23 16:59:06 UTC

I ran many iterations of the split image test cases, and although there were still problems like bug 733114 occuring, I no longer saw the "LV in use: not deactivating; Failed to deactivate former mirror image" error. Marking verified with the latest rpms.

2.6.32-198.el6.x86_64

lvm2-2.02.87-3.el6    BUILT: Wed Sep 21 09:54:55 CDT 2011
lvm2-libs-2.02.87-3.el6    BUILT: Wed Sep 21 09:54:55 CDT 2011
lvm2-cluster-2.02.87-3.el6    BUILT: Wed Sep 21 09:54:55 CDT 2011
udev-147-2.38.el6    BUILT: Fri Sep  9 16:25:50 CDT 2011
device-mapper-1.02.66-3.el6    BUILT: Wed Sep 21 09:54:55 CDT 2011
device-mapper-libs-1.02.66-3.el6    BUILT: Wed Sep 21 09:54:55 CDT 2011
device-mapper-event-1.02.66-3.el6    BUILT: Wed Sep 21 09:54:55 CDT 2011
device-mapper-event-libs-1.02.66-3.el6    BUILT: Wed Sep 21 09:54:55 CDT 2011
cmirror-2.02.87-3.el6    BUILT: Wed Sep 21 09:54:55 CDT 2011

Comment 6 Peter Rajnoha 2011-10-26 13:01:47 UTC

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Do not document.

Comment 7 errata-xmlrpc 2011-12-06 17:03:15 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1522.html