Bug 677485

Summary: deactivating an exclusively activated cmirror with snapshot, leaves mlog device still activated
Product: Red Hat Enterprise Linux 6 Reporter: Corey Marthaler <cmarthal>
Component: lvm2Assignee: Jonathan Earl Brassow <jbrassow>
Status: CLOSED WONTFIX QA Contact: Corey Marthaler <cmarthal>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.1CC: agk, dwysocha, heinzm, jbrassow, mbroz, prajnoha, prockai, thornber, zkabelac
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-09-16 14:59:02 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Corey Marthaler 2011-02-14 23:42:30 UTC
Description of problem:
The deactivation at the end of this test case continues to leave the mlog device active.


================================================================================
Iteration 0.1 started at Mon Feb 14 16:34:38 CST 2011
================================================================================
Scenario: Kill primary leg of synced 2 leg mirror(s)

********* Mirror hash info for this scenario *********
* names:              syncd_primary_2legs_1
* sync:               1
* leg devices:        /dev/sdg1 /dev/sdd1
* log devices:        /dev/sdh1
* no MDA devices:     
* failpv(s):          /dev/sdg1
* failnode(s):        taft-01
* additional snap:    /dev/sdd1
* leg fault policy:   remove
* log fault policy:   allocate
******************************************************

Creating mirror(s) on taft-01...
taft-01: lvcreate -m 1 -n syncd_primary_2legs_1 -L 600M helter_skelter /dev/sdg1:0-1000 /dev/sdd1:0-1000 /dev/sdh1:0-150
EXCLUSIVELY ACTIVATING MIRROR FOR CSNAP CREATION
Creating a snapshot volume of each of the mirrors

Waiting until all mirrors become fully syncd...
   0/1 mirror(s) are fully synced: ( 89.58% )
   1/1 mirror(s) are fully synced: ( 100.00% )

Creating ext on top of mirror(s) on taft-01...
mke2fs 1.41.12 (17-May-2010)
Mounting mirrored ext filesystems on taft-01...

Writing verification files (checkit) to mirror(s) on...
    ---- taft-01 ----

Sleeping 10 seconds to get some outsanding EXT I/O locks before the failure 
Verifying files (checkit) on mirror(s) on...
    ---- taft-01 ----

Disabling device sdg on taft-01

Attempting I/O to cause mirror down conversion(s) on taft-01
10+0 records in
10+0 records out
41943040 bytes (42 MB) copied, 0.233696 s, 179 MB/s

Verifying current sanity of lvm after the failure
Verifying FAILED device /dev/sdg1 is *NOT* in the volume(s)
Verifying LOG device(s) /dev/sdh1 are *NOT* in the mirror/linear(s)
Verifying LEG device /dev/sdd1 *IS* in the volume(s)
verify the dm devices associated with /dev/sdg1 have been removed as expected
Checking REMOVAL of syncd_primary_2legs_1_mimage_0 on:  taft-01

Verify that each of the mirror repairs finished successfully

Verifying files (checkit) on mirror(s) on...
    ---- taft-01 ----

Enabling device sdg on taft-01

Recreating PVs /dev/sdg1
  WARNING: Volume group helter_skelter is not consistent
Extending the recreated PVs back into VG helter_skelter

Attempting to reconvert volume(s) back to their original mirrored state on taft-01...
(it's possible this may be a redundant operation in some allocate scenarios)
********************************************************
*** HACK for inability to upconvert snapped mirrors *** 
*** HACK for inability to upconvert from disk->mirrored logs *** 
********************************************************
taft-01: lvconvert -m 0 helter_skelter/syncd_primary_2legs_1
  Logical volume syncd_primary_2legs_1 is already not mirrored.
taft-01: lvconvert --mirrorlog disk -m 1 -b helter_skelter/syncd_primary_2legs_1 /dev/sdg1:0-1000 /dev/sdd1:0-1000 /dev/sdh1:0-150
Verifying the up conversions by checking if all original leg devices are back in the mirror(s)
Verifying device /dev/sdg1 *IS* one of the legs in the mirror(s)
Verifying device /dev/sdd1 *IS* one of the legs in the mirror(s)
Checking for leftover '-missing_0_0' or 'unknown devices'

Waiting until all mirrors become fully syncd...
   1/1 mirror(s) are fully synced: ( 100.00% )

Verifying files (checkit) on mirror(s) on...
        ---- taft-01 ----

Stopping the io load (collie/xdoio) on mirror(s)
<killed name="taft-01_syncd_primary_2legs_1" pid="13573" time="Mon Feb 14 16:36:38 2011" type="cmd" duration="80" signal="2" />
Unmounting ext and removing mnt point on taft-01...

Deactivating and removing mirror(s)
lvchange -an /dev/helter_skelter/syncd_primary_2legs_1
  Error locking on node taft-01: Unable to deactivate open helter_skelter-syncd_primary_2legs_1_mlog (253:3)

[DEACTIVATE FAILED]


[root@taft-01 ~]# lvs -a -o +devices
 LV                               VG             Attr   LSize   Origin                Snap%  Log                        Copy%  Devices
 snap0                            helter_skelter swi--- 252.00m syncd_primary_2legs_1                                          /dev/sdd1(150)
 syncd_primary_2legs_1            helter_skelter owi--- 600.00m                              syncd_primary_2legs_1_mlog        syncd_primary_2legs_1_mimage_0(0),syncd_primary_2legs_1_mimage_1(0)
 [syncd_primary_2legs_1_mimage_0] helter_skelter Iwi--- 600.00m                                                                /dev/sdd1(0)
 [syncd_primary_2legs_1_mimage_1] helter_skelter Iwi--- 600.00m                                                                /dev/sdg1(0)
 [syncd_primary_2legs_1_mlog]     helter_skelter lwi-a-   4.00m                                                                /dev/sdh1(0)

[root@taft-01 ~]# dmsetup ls
helter_skelter-syncd_primary_2legs_1_mlog       (253, 3)


Version-Release number of selected component (if applicable):
2.6.32-94.el6.x86_64

lvm2-2.02.83-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
lvm2-libs-2.02.83-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
lvm2-cluster-2.02.83-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
udev-147-2.31.el6    BUILT: Wed Jan 26 05:39:15 CST 2011
device-mapper-1.02.62-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
device-mapper-libs-1.02.62-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
device-mapper-event-1.02.62-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
device-mapper-event-libs-1.02.62-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011
cmirror-2.02.83-2.el6    BUILT: Tue Feb  8 10:10:57 CST 2011


How reproducible:
Most of the time

Comment 1 RHEL Program Management 2011-04-04 01:58:25 UTC
Since RHEL 6.1 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.

Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.

Comment 2 Alasdair Kergon 2011-06-09 14:55:29 UTC
Is this udev again?  Does upstream code manage any better?

Comment 3 Zdenek Kabelac 2011-08-31 09:49:38 UTC
Jon is the mirror code properly synchronizing with udev?

Comment 4 Jonathan Earl Brassow 2011-09-16 14:59:02 UTC
Snapshots of mirrors (cluster or otherwise) is not fully supported.  I will never be fully supported.

Snapshots of mirrors will have to wait for the next mirror implementation (i.e. "raid1") to be complete.