Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
DescriptionCorey Marthaler
2011-02-14 23:42:30 UTC
Description of problem:
The deactivation at the end of this test case continues to leave the mlog device active.
================================================================================
Iteration 0.1 started at Mon Feb 14 16:34:38 CST 2011
================================================================================
Scenario: Kill primary leg of synced 2 leg mirror(s)
********* Mirror hash info for this scenario *********
* names: syncd_primary_2legs_1
* sync: 1
* leg devices: /dev/sdg1 /dev/sdd1
* log devices: /dev/sdh1
* no MDA devices:
* failpv(s): /dev/sdg1
* failnode(s): taft-01
* additional snap: /dev/sdd1
* leg fault policy: remove
* log fault policy: allocate
******************************************************
Creating mirror(s) on taft-01...
taft-01: lvcreate -m 1 -n syncd_primary_2legs_1 -L 600M helter_skelter /dev/sdg1:0-1000 /dev/sdd1:0-1000 /dev/sdh1:0-150
EXCLUSIVELY ACTIVATING MIRROR FOR CSNAP CREATION
Creating a snapshot volume of each of the mirrors
Waiting until all mirrors become fully syncd...
0/1 mirror(s) are fully synced: ( 89.58% )
1/1 mirror(s) are fully synced: ( 100.00% )
Creating ext on top of mirror(s) on taft-01...
mke2fs 1.41.12 (17-May-2010)
Mounting mirrored ext filesystems on taft-01...
Writing verification files (checkit) to mirror(s) on...
---- taft-01 ----
Sleeping 10 seconds to get some outsanding EXT I/O locks before the failure
Verifying files (checkit) on mirror(s) on...
---- taft-01 ----
Disabling device sdg on taft-01
Attempting I/O to cause mirror down conversion(s) on taft-01
10+0 records in
10+0 records out
41943040 bytes (42 MB) copied, 0.233696 s, 179 MB/s
Verifying current sanity of lvm after the failure
Verifying FAILED device /dev/sdg1 is *NOT* in the volume(s)
Verifying LOG device(s) /dev/sdh1 are *NOT* in the mirror/linear(s)
Verifying LEG device /dev/sdd1 *IS* in the volume(s)
verify the dm devices associated with /dev/sdg1 have been removed as expected
Checking REMOVAL of syncd_primary_2legs_1_mimage_0 on: taft-01
Verify that each of the mirror repairs finished successfully
Verifying files (checkit) on mirror(s) on...
---- taft-01 ----
Enabling device sdg on taft-01
Recreating PVs /dev/sdg1
WARNING: Volume group helter_skelter is not consistent
Extending the recreated PVs back into VG helter_skelter
Attempting to reconvert volume(s) back to their original mirrored state on taft-01...
(it's possible this may be a redundant operation in some allocate scenarios)
********************************************************
*** HACK for inability to upconvert snapped mirrors ***
*** HACK for inability to upconvert from disk->mirrored logs ***
********************************************************
taft-01: lvconvert -m 0 helter_skelter/syncd_primary_2legs_1
Logical volume syncd_primary_2legs_1 is already not mirrored.
taft-01: lvconvert --mirrorlog disk -m 1 -b helter_skelter/syncd_primary_2legs_1 /dev/sdg1:0-1000 /dev/sdd1:0-1000 /dev/sdh1:0-150
Verifying the up conversions by checking if all original leg devices are back in the mirror(s)
Verifying device /dev/sdg1 *IS* one of the legs in the mirror(s)
Verifying device /dev/sdd1 *IS* one of the legs in the mirror(s)
Checking for leftover '-missing_0_0' or 'unknown devices'
Waiting until all mirrors become fully syncd...
1/1 mirror(s) are fully synced: ( 100.00% )
Verifying files (checkit) on mirror(s) on...
---- taft-01 ----
Stopping the io load (collie/xdoio) on mirror(s)
<killed name="taft-01_syncd_primary_2legs_1" pid="13573" time="Mon Feb 14 16:36:38 2011" type="cmd" duration="80" signal="2" />
Unmounting ext and removing mnt point on taft-01...
Deactivating and removing mirror(s)
lvchange -an /dev/helter_skelter/syncd_primary_2legs_1
Error locking on node taft-01: Unable to deactivate open helter_skelter-syncd_primary_2legs_1_mlog (253:3)
[DEACTIVATE FAILED]
[root@taft-01 ~]# lvs -a -o +devices
LV VG Attr LSize Origin Snap% Log Copy% Devices
snap0 helter_skelter swi--- 252.00m syncd_primary_2legs_1 /dev/sdd1(150)
syncd_primary_2legs_1 helter_skelter owi--- 600.00m syncd_primary_2legs_1_mlog syncd_primary_2legs_1_mimage_0(0),syncd_primary_2legs_1_mimage_1(0)
[syncd_primary_2legs_1_mimage_0] helter_skelter Iwi--- 600.00m /dev/sdd1(0)
[syncd_primary_2legs_1_mimage_1] helter_skelter Iwi--- 600.00m /dev/sdg1(0)
[syncd_primary_2legs_1_mlog] helter_skelter lwi-a- 4.00m /dev/sdh1(0)
[root@taft-01 ~]# dmsetup ls
helter_skelter-syncd_primary_2legs_1_mlog (253, 3)
Version-Release number of selected component (if applicable):
2.6.32-94.el6.x86_64
lvm2-2.02.83-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011
lvm2-libs-2.02.83-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011
lvm2-cluster-2.02.83-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011
udev-147-2.31.el6 BUILT: Wed Jan 26 05:39:15 CST 2011
device-mapper-1.02.62-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011
device-mapper-libs-1.02.62-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011
device-mapper-event-1.02.62-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011
device-mapper-event-libs-1.02.62-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011
cmirror-2.02.83-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011
How reproducible:
Most of the time
Comment 1RHEL Program Management
2011-04-04 01:58:25 UTC
Since RHEL 6.1 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.
Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.
Jon is the mirror code properly synchronizing with udev?
Comment 4Jonathan Earl Brassow
2011-09-16 14:59:02 UTC
Snapshots of mirrors (cluster or otherwise) is not fully supported. I will never be fully supported.
Snapshots of mirrors will have to wait for the next mirror implementation (i.e. "raid1") to be complete.
Description of problem: The deactivation at the end of this test case continues to leave the mlog device active. ================================================================================ Iteration 0.1 started at Mon Feb 14 16:34:38 CST 2011 ================================================================================ Scenario: Kill primary leg of synced 2 leg mirror(s) ********* Mirror hash info for this scenario ********* * names: syncd_primary_2legs_1 * sync: 1 * leg devices: /dev/sdg1 /dev/sdd1 * log devices: /dev/sdh1 * no MDA devices: * failpv(s): /dev/sdg1 * failnode(s): taft-01 * additional snap: /dev/sdd1 * leg fault policy: remove * log fault policy: allocate ****************************************************** Creating mirror(s) on taft-01... taft-01: lvcreate -m 1 -n syncd_primary_2legs_1 -L 600M helter_skelter /dev/sdg1:0-1000 /dev/sdd1:0-1000 /dev/sdh1:0-150 EXCLUSIVELY ACTIVATING MIRROR FOR CSNAP CREATION Creating a snapshot volume of each of the mirrors Waiting until all mirrors become fully syncd... 0/1 mirror(s) are fully synced: ( 89.58% ) 1/1 mirror(s) are fully synced: ( 100.00% ) Creating ext on top of mirror(s) on taft-01... mke2fs 1.41.12 (17-May-2010) Mounting mirrored ext filesystems on taft-01... Writing verification files (checkit) to mirror(s) on... ---- taft-01 ---- Sleeping 10 seconds to get some outsanding EXT I/O locks before the failure Verifying files (checkit) on mirror(s) on... ---- taft-01 ---- Disabling device sdg on taft-01 Attempting I/O to cause mirror down conversion(s) on taft-01 10+0 records in 10+0 records out 41943040 bytes (42 MB) copied, 0.233696 s, 179 MB/s Verifying current sanity of lvm after the failure Verifying FAILED device /dev/sdg1 is *NOT* in the volume(s) Verifying LOG device(s) /dev/sdh1 are *NOT* in the mirror/linear(s) Verifying LEG device /dev/sdd1 *IS* in the volume(s) verify the dm devices associated with /dev/sdg1 have been removed as expected Checking REMOVAL of syncd_primary_2legs_1_mimage_0 on: taft-01 Verify that each of the mirror repairs finished successfully Verifying files (checkit) on mirror(s) on... ---- taft-01 ---- Enabling device sdg on taft-01 Recreating PVs /dev/sdg1 WARNING: Volume group helter_skelter is not consistent Extending the recreated PVs back into VG helter_skelter Attempting to reconvert volume(s) back to their original mirrored state on taft-01... (it's possible this may be a redundant operation in some allocate scenarios) ******************************************************** *** HACK for inability to upconvert snapped mirrors *** *** HACK for inability to upconvert from disk->mirrored logs *** ******************************************************** taft-01: lvconvert -m 0 helter_skelter/syncd_primary_2legs_1 Logical volume syncd_primary_2legs_1 is already not mirrored. taft-01: lvconvert --mirrorlog disk -m 1 -b helter_skelter/syncd_primary_2legs_1 /dev/sdg1:0-1000 /dev/sdd1:0-1000 /dev/sdh1:0-150 Verifying the up conversions by checking if all original leg devices are back in the mirror(s) Verifying device /dev/sdg1 *IS* one of the legs in the mirror(s) Verifying device /dev/sdd1 *IS* one of the legs in the mirror(s) Checking for leftover '-missing_0_0' or 'unknown devices' Waiting until all mirrors become fully syncd... 1/1 mirror(s) are fully synced: ( 100.00% ) Verifying files (checkit) on mirror(s) on... ---- taft-01 ---- Stopping the io load (collie/xdoio) on mirror(s) <killed name="taft-01_syncd_primary_2legs_1" pid="13573" time="Mon Feb 14 16:36:38 2011" type="cmd" duration="80" signal="2" /> Unmounting ext and removing mnt point on taft-01... Deactivating and removing mirror(s) lvchange -an /dev/helter_skelter/syncd_primary_2legs_1 Error locking on node taft-01: Unable to deactivate open helter_skelter-syncd_primary_2legs_1_mlog (253:3) [DEACTIVATE FAILED] [root@taft-01 ~]# lvs -a -o +devices LV VG Attr LSize Origin Snap% Log Copy% Devices snap0 helter_skelter swi--- 252.00m syncd_primary_2legs_1 /dev/sdd1(150) syncd_primary_2legs_1 helter_skelter owi--- 600.00m syncd_primary_2legs_1_mlog syncd_primary_2legs_1_mimage_0(0),syncd_primary_2legs_1_mimage_1(0) [syncd_primary_2legs_1_mimage_0] helter_skelter Iwi--- 600.00m /dev/sdd1(0) [syncd_primary_2legs_1_mimage_1] helter_skelter Iwi--- 600.00m /dev/sdg1(0) [syncd_primary_2legs_1_mlog] helter_skelter lwi-a- 4.00m /dev/sdh1(0) [root@taft-01 ~]# dmsetup ls helter_skelter-syncd_primary_2legs_1_mlog (253, 3) Version-Release number of selected component (if applicable): 2.6.32-94.el6.x86_64 lvm2-2.02.83-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011 lvm2-libs-2.02.83-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011 lvm2-cluster-2.02.83-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011 udev-147-2.31.el6 BUILT: Wed Jan 26 05:39:15 CST 2011 device-mapper-1.02.62-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011 device-mapper-libs-1.02.62-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011 device-mapper-event-1.02.62-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011 device-mapper-event-libs-1.02.62-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011 cmirror-2.02.83-2.el6 BUILT: Tue Feb 8 10:10:57 CST 2011 How reproducible: Most of the time