Bug 227155 - down converting cmirrors during I/O can lead to the log devices staying active
Summary: down converting cmirrors during I/O can lead to the log devices staying active
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Red Hat Cluster Suite
Classification: Retired
Component: cmirror
Version: 4
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Jonathan Earl Brassow
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On: 217895
Blocks:
TreeView+ depends on / blocked
 
Reported: 2007-02-02 20:18 UTC by Corey Marthaler
Modified: 2010-01-12 02:02 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2008-06-30 20:17:10 UTC
Embargoed:


Attachments (Terms of Use)

Description Corey Marthaler 2007-02-02 20:18:55 UTC
Description of problem:
This issues may have to do with bz 225487. I was running GFS I/O to filesystems
on top of 3 cmirrors and was down and up converting them, and while doing so,
sometimes the mlogs do not get deactivated or removed and that results in the
mirror (now a linear) not being able to be reconverted back into a mirror again.


[root@link-02 ~]# lvscan
  ACTIVE            '/dev/vg/mirror1' [5.00 GB] inherit
  ACTIVE            '/dev/vg/mirror2' [5.00 GB] inherit
  ACTIVE            '/dev/vg/mirror3' [5.00 GB] inherit
  ACTIVE            '/dev/vg/mirror2_mlog' [4.00 MB] inherit
  ACTIVE            '/dev/vg/mirror3_mlog' [4.00 MB] inherit

[root@link-02 ~]# lvs -a -o +devices
  LV                 VG   Attr   LSize Origin Snap%  Move Log          Copy% 
Devices                   
  mirror1            vg   Mwi-ao 5.00G                    mirror1_mlog 100.00
mirror1_mimage_0(0),mirror1_mimage_1(0)
  [mirror1_mimage_0] vg   iwi-ao 5.00G                                       
/dev/sdh1(1280)           
  [mirror1_mimage_1] vg   iwi-ao 5.00G                                       
/dev/sdb1(0)              
  [mirror1_mlog]     vg   lwi-ao 4.00M                                       
/dev/sda1(1290)           
  mirror2            vg   -wi-ao 5.00G                                       
/dev/sdh1(2560)           
  mirror2_mlog       vg   -wi-a- 4.00M                                       
/dev/sda1(1288)           
  mirror3            vg   -wi-ao 5.00G                                       
/dev/sdh1(3840)           
  mirror3_mlog       vg   -wi-a- 4.00M                                       
/dev/sda1(1289)           

[root@link-02 ~]# lvconvert -m 1 vg/mirror2
  Internal error: Duplicate LV name mirror2_mlog detected in vg.
  Failed to create mirror log.

[root@link-02 ~]# dmsetup ls
vg-mirror1_mimage_1     (253, 4)
vg-mirror1_mimage_0     (253, 3)
vg-mirror1_mlog (253, 2)
vg-mirror3      (253, 13)
vg-mirror2_mlog (253, 6)
vg-mirror2      (253, 9)
vg-mirror3_mlog (253, 7)
vg-mirror1      (253, 5)
VolGroup00-LogVol01     (253, 1)
VolGroup00-LogVol00     (253, 0)



Version-Release number of selected component (if applicable):
2.6.9-45.ELsmp
lvm2-2.02.21-1.el4
lvm2-cluster-2.02.21-1.el4
device-mapper-1.02.17-1.el4
cmirror-1.0.1-1
cmirror-kernel-smp-2.6.9-18.9

Comment 1 Jonathan Earl Brassow 2007-02-02 21:08:08 UTC
I suspect that this is related to Ben's issue where dmeventd was having
comunication problems.

Try device-mapper-1.02.17-2.el4


Comment 2 Corey Marthaler 2007-04-11 16:33:28 UTC
Have not seen this issue in awhile, marking this verified.

Comment 3 Corey Marthaler 2007-08-30 19:51:11 UTC
I just hit this bug while doing the same type of mirror convert operations
during I/O originally listed in the bug. 


while true; do lvconvert -m 0 vg1/mirror1; lvs -a -o +devices; sleep 10;
lvconvert -m 1 vg1/mirror1; sleep 30; lvs -a -o +devices; done 


  LV           VG         Attr   LSize  Origin Snap%  Move Log Copy%  Devices
  LogVol00     VolGroup00 -wi-ao 35.16G                               /dev/hda2(0)
  LogVol01     VolGroup00 -wi-ao  1.94G                              
/dev/hda2(1125)
  mirror1      vg1        -wi-ao 18.00G                               /dev/sdh1(0)
  mirror1_mlog vg1        -wi-a-  4.00M                               /dev/sdb1(0)
  mirror2      vg2        -wi-ao 18.00G                               /dev/sdh2(0)
  mirror2_mlog vg2        -wi-a-  4.00M                               /dev/sdd1(0)
  mirror3      vg3        -wi-ao 18.00G                               /dev/sdh3(0)
  Internal error: Duplicate LV name mirror1_mlog detected in vg1.
  Failed to create mirror log.

I was running with:
lvm2-2.02.27-1.el4
lvm2-cluster-2.02.27-1.el4
device-mapper-1.02.21-1.el4

Comment 4 Corey Marthaler 2007-09-05 16:34:21 UTC
Reproduced this with the following as well...
lvm2-cluster-2.02.27-2.el4
lvm2-2.02.27-2.el4


Comment 5 Corey Marthaler 2007-09-05 16:44:51 UTC
This bug may be related to bz 236289. I saw similar stuff after reproducing this
bug today. Internal errors on one node and locking problems on the other. This
was while doing up and down converts.

Link-02:
  Internal error: Duplicate LV name mirror1_mlog detected in vg1.
  Failed to create mirror log.

Link-07:
  Error locking on node link-02: Command timed out
  Failed to lock mirror2

Link-08:
  Error locking on node link-02: Command timed out
  Problem reactivating mirror3


Comment 6 Jonathan Earl Brassow 2007-10-01 15:55:04 UTC
I haven't seen this in a while.  The team has been changing various aspects of
'dmeventd'.  Some of those changes have caused this issue to show up from time
to time.  I'm wondering where we stand now...

assigned -> needinfo

Comment 7 Corey Marthaler 2007-12-20 19:53:06 UTC
Unable to verify this fix due to bz 217895.

Comment 8 Jonathan Earl Brassow 2008-02-15 21:27:26 UTC
Adding that this bug depends on 217895 in order to be verified.

Comment 9 Jonathan Earl Brassow 2008-04-03 15:18:15 UTC
This one should be in modified...

Comment 10 Corey Marthaler 2008-06-30 20:17:10 UTC
Marking this bug verified.


Note You need to log in before you can comment on or make changes to this bug.