Bug 243754 - cmirror down covert attempt during a mount can cause failures
cmirror down covert attempt during a mount can cause failures
Status: CLOSED CURRENTRELEASE
Product: Red Hat Cluster Suite
Classification: Red Hat
Component: cmirror (Show other bugs)
4
All Linux
low Severity low
: ---
: ---
Assigned To: Jonathan Earl Brassow
Cluster QE
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2007-06-11 14:26 EDT by Corey Marthaler
Modified: 2010-04-27 11:01 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-04-27 11:01:29 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Corey Marthaler 2007-06-11 14:26:43 EDT
Description of problem:
I was testing how long it takes to mount cmirrors that are syncing due to up
converts (and there may be a bug in there as well cuz it takes a long time) and
 while I was waiting, I attemped to down conver the cmirror back to the linear
that it was, and that failed resulting in the mount cmd hanging, as well as
syncing deadlocked.

# here mirror3 is a linear
[root@link-07 ~]# lvs -a -o +devices
  LV                 VG         Attr   LSize  Origin Snap%  Move Log         
Copy%  Devices
  LogVol00           VolGroup00 -wi-ao 72.44G                                  
     /dev/hda2(0)
  LogVol01           VolGroup00 -wi-ao  1.94G                                  
     /dev/hda2(2318)
  mirror1            corey1     Mwi-a- 10.00G                    mirror1_mlog
100.00 mirror1_mimage_0(0),mirror1_mimage_1(0)
  [mirror1_mimage_0] corey1     iwi-ao 10.00G                                  
     /dev/sda1(0)
  [mirror1_mimage_1] corey1     iwi-ao 10.00G                                  
     /dev/sdb1(0)
  [mirror1_mlog]     corey1     lwi-ao  4.00M                                  
     /dev/sdb2(0)
  mirror2            corey2     Mwi-a- 10.00G                                
100.00 mirror2_mimage_0(0),mirror2_mimage_1(0)
  [mirror2_mimage_0] corey2     iwi-ao 10.00G                                  
     /dev/sdd1(0)
  [mirror2_mimage_1] corey2     iwi-ao 10.00G                                  
     /dev/sdc1(0)
  mirror3            corey3     -wi-a- 10.00G                                  
     /dev/sdg1(0)


[root@link-07 ~]# lvconvert -m 1 --corelog  corey3/mirror3
  Logical volume mirror3 converted.
[root@link-07 ~]# mount -t gfs /dev/corey3/mirror3 /mnt/mirror3

# while the mount cmd is taking a long time, I decide to down convert it back to
a linear on link-08
[root@link-08 ~]# lvconvert -m 0 corey3/mirror3
  Error locking on node link-07: Command timed out
  Failed to lock mirror3
  Logical volume mirror3 converted.
[root@link-08 ~]# lvs -a -o +devices
  LV                 VG         Attr   LSize  Origin Snap%  Move Log         
Copy%  Devices
  LogVol00           VolGroup00 -wi-ao 72.44G                                  
     /dev/hda2(0)
  LogVol01           VolGroup00 -wi-ao  1.94G                                  
     /dev/hda2(2318)
  mirror1            corey1     Mwi-a- 10.00G                    mirror1_mlog
100.00 mirror1_mimage_0(0),mirror1_mimage_1(0)
  [mirror1_mimage_0] corey1     iwi-ao 10.00G                                  
     /dev/sda1(0)
  [mirror1_mimage_1] corey1     iwi-ao 10.00G                                  
     /dev/sdb1(0)
  [mirror1_mlog]     corey1     lwi-ao  4.00M                                  
     /dev/sdb2(0)
  mirror2            corey2     Mwi-a- 10.00G                                
100.00 mirror2_mimage_0(0),mirror2_mimage_1(0)
  [mirror2_mimage_0] corey2     iwi-ao 10.00G                                  
     /dev/sdd1(0)
  [mirror2_mimage_1] corey2     iwi-ao 10.00G                                  
     /dev/sdc1(0)
  mirror3            corey3     Mwi-s- 10.00G                                  
3.91 mirror3_mimage_0(0),mirror3_mimage_1(0)
  [mirror3_mimage_0] corey3     iwi-so 10.00G                                  
     /dev/sdg1(0)
  [mirror3_mimage_1] corey3     iwi-so 10.00G                                  
     /dev/sdf1(0)
[root@link-08 ~]# lvs -a -o +devices
  LV                 VG         Attr   LSize  Origin Snap%  Move Log         
Copy%  Devices
  LogVol00           VolGroup00 -wi-ao 72.44G                                  
     /dev/hda2(0)
  LogVol01           VolGroup00 -wi-ao  1.94G                                  
     /dev/hda2(2318)
  mirror1            corey1     Mwi-a- 10.00G                    mirror1_mlog
100.00 mirror1_mimage_0(0),mirror1_mimage_1(0)
  [mirror1_mimage_0] corey1     iwi-ao 10.00G                                  
     /dev/sda1(0)
  [mirror1_mimage_1] corey1     iwi-ao 10.00G                                  
     /dev/sdb1(0)
  [mirror1_mlog]     corey1     lwi-ao  4.00M                                  
     /dev/sdb2(0)
  mirror2            corey2     Mwi-a- 10.00G                                
100.00 mirror2_mimage_0(0),mirror2_mimage_1(0)
  [mirror2_mimage_0] corey2     iwi-ao 10.00G                                  
     /dev/sdd1(0)
  [mirror2_mimage_1] corey2     iwi-ao 10.00G                                  
     /dev/sdc1(0)
  mirror3            corey3     Mwi-s- 10.00G                                  
3.91 mirror3_mimage_0(0),mirror3_mimage_1(0)
  [mirror3_mimage_0] corey3     iwi-so 10.00G                                  
     /dev/sdg1(0)
  [mirror3_mimage_1] corey3     iwi-so 10.00G                                  
     /dev/sdf1(0)





[root@link-07 ~]# cman_tool services
Service          Name                              GID LID State     Code
Fence Domain:    "default"                           2   2 run       -
[1 2 4 3]

DLM Lock Space:  "clvmd"                             3   3 run       -
[1 2 3 4]

DLM Lock Space:  "clustered_log"                     4   4 run       -
[1 2 3 4]

DLM Lock Space:  "3"                                31  13 run       -
[1]

GFS Mount Group: "3"                                32  14 run       -
[1]

[root@link-07 ~]# dmsetup ls
corey1-mirror1_mimage_1 (253, 7)
corey1-mirror1  (253, 8)
corey1-mirror1_mimage_0 (253, 6)
corey2-mirror2_mimage_1 (253, 3)
corey2-mirror2_mimage_0 (253, 2)
corey2-mirror2  (253, 4)
corey3-mirror3_mimage_1 (253, 10)
corey3-mirror3_mimage_0 (253, 9)
VolGroup00-LogVol01     (253, 1)
corey3-mirror3  (253, 12)
VolGroup00-LogVol00     (253, 0)
corey1-mirror1_mlog     (253, 5)
[root@link-07 ~]# dmsetup info
Name:              corey1-mirror1_mimage_1
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 7
Number of targets: 1
UUID: LVM-0QLi8UKjRguEwy4KI1k9KriBg4lyb2j1V9V23NFX7ZxmKdVvCMdSh2B0Q5uPTSKB

Name:              corey1-mirror1
State:             ACTIVE
Tables present:    LIVE
Open count:        0
Event number:      1
Major, minor:      253, 8
Number of targets: 1
UUID: LVM-0QLi8UKjRguEwy4KI1k9KriBg4lyb2j1fydUC356OtfUZVfh74E4yWHGepLEAQu5

Name:              corey1-mirror1_mimage_0
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 6
Number of targets: 1
UUID: LVM-0QLi8UKjRguEwy4KI1k9KriBg4lyb2j1YjGdtpLiSDkmhud2mVjyNgZveaSecV87

Name:              corey2-mirror2_mimage_1
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 3
Number of targets: 1
UUID: LVM-caAVDd2dPhYq3rRmtemqUvgAt90oxO1vpnsQHYANmpEyo0yXJQwqe62JRBXRDtT9

Name:              corey2-mirror2_mimage_0
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 2
Number of targets: 1
UUID: LVM-caAVDd2dPhYq3rRmtemqUvgAt90oxO1vczGS49XAjOu44xh1tr4PQr7WiR5ErZl3

Name:              corey2-mirror2
State:             ACTIVE
Tables present:    LIVE
Open count:        0
Event number:      1
Major, minor:      253, 4
Number of targets: 1
UUID: LVM-caAVDd2dPhYq3rRmtemqUvgAt90oxO1vC3HFk9CLB9ELCxhSZBOf1lkkE11Rdz25

Name:              corey3-mirror3_mimage_1
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 10
Number of targets: 1
UUID: LVM-vCWvB54l0xEiiQuUH1anc8UJwd1KeH2RCPDWgbqT9UuFK4p25MUvzxtMCyqNu3uF

Name:              corey3-mirror3_mimage_0
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 9
Number of targets: 1
UUID: LVM-vCWvB54l0xEiiQuUH1anc8UJwd1KeH2RJELilE98AuY6yV853GNVoL1XNmkDpTaK

Name:              VolGroup00-LogVol01
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 1
Number of targets: 1
UUID: LVM-8900P3byCnuwn8bRCKduIK0tDMwWj4Ktv37kwLqojWwuLYSJJ6K5CSxKbZfCVMQi

Name:              corey3-mirror3
State:             ACTIVE
Tables present:    LIVE & INACTIVE
Open count:        1
Event number:      0
Major, minor:      253, 12
Number of targets: 1
UUID: LVM-vCWvB54l0xEiiQuUH1anc8UJwd1KeH2RrVxFHPheVHLeYGToOyHqciYLB1QRho7o

Name:              VolGroup00-LogVol00
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 0
Number of targets: 1
UUID: LVM-8900P3byCnuwn8bRCKduIK0tDMwWj4Kt8qx2Ghea150aw3oysoj9Znz4f5UhYSOq

Name:              corey1-mirror1_mlog
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 5
Number of targets: 1
UUID: LVM-0QLi8UKjRguEwy4KI1k9KriBg4lyb2j1hCJh1EVAOxhYcsIkh5Fwap1yOkxFeidY

[root@link-07 ~]# dmsetup status
corey1-mirror1_mimage_1: 0 20971520 linear
corey1-mirror1: 0 20971520 mirror 2 253:6 253:7 20480/20480 1 AA 3
clustered_disk 253:5 A
corey1-mirror1_mimage_0: 0 20971520 linear
corey2-mirror2_mimage_1: 0 20971520 linear
corey2-mirror2_mimage_0: 0 20971520 linear
corey2-mirror2: 0 20971520 mirror 2 253:2 253:3 20480/20480 1 AA 1 clustered_core
corey3-mirror3_mimage_1: 0 20971520 linear
corey3-mirror3_mimage_0: 0 20971520 linear
VolGroup00-LogVol01: 0 4063232 linear
corey3-mirror3: 0 20971520 mirror 2 253:9 253:10 804/20480 1 AA 1 clustered_core
VolGroup00-LogVol00: 0 151912448 linear
corey1-mirror1_mlog: 0 8192 linear





[root@link-08 ~]# cman_tool services
Service          Name                              GID LID State     Code
Fence Domain:    "default"                           2   2 run       -
[1 2 4 3]

DLM Lock Space:  "clvmd"                             3   3 run       -
[1 2 3 4]

DLM Lock Space:  "clustered_log"                     4   4 run       -
[1 2 3 4]

[root@link-08 ~]# dmsetup ls
corey1-mirror1_mimage_1 (253, 7)
corey1-mirror1  (253, 8)
corey1-mirror1_mimage_0 (253, 6)
corey2-mirror2_mimage_1 (253, 3)
corey2-mirror2_mimage_0 (253, 2)
corey2-mirror2  (253, 4)
corey3-mirror3_mimage_1 (253, 10)
corey3-mirror3_mimage_0 (253, 9)
VolGroup00-LogVol01     (253, 1)
corey3-mirror3  (253, 12)
VolGroup00-LogVol00     (253, 0)
corey1-mirror1_mlog     (253, 5)
[root@link-08 ~]# dmsetup info
Name:              corey1-mirror1_mimage_1
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 7
Number of targets: 1
UUID: LVM-0QLi8UKjRguEwy4KI1k9KriBg4lyb2j1V9V23NFX7ZxmKdVvCMdSh2B0Q5uPTSKB

Name:              corey1-mirror1
State:             ACTIVE
Tables present:    LIVE
Open count:        0
Event number:      1
Major, minor:      253, 8
Number of targets: 1
UUID: LVM-0QLi8UKjRguEwy4KI1k9KriBg4lyb2j1fydUC356OtfUZVfh74E4yWHGepLEAQu5

Name:              corey1-mirror1_mimage_0
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 6
Number of targets: 1
UUID: LVM-0QLi8UKjRguEwy4KI1k9KriBg4lyb2j1YjGdtpLiSDkmhud2mVjyNgZveaSecV87

Name:              corey2-mirror2_mimage_1
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 3
Number of targets: 1
UUID: LVM-caAVDd2dPhYq3rRmtemqUvgAt90oxO1vpnsQHYANmpEyo0yXJQwqe62JRBXRDtT9

Name:              corey2-mirror2_mimage_0
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 2
Number of targets: 1
UUID: LVM-caAVDd2dPhYq3rRmtemqUvgAt90oxO1vczGS49XAjOu44xh1tr4PQr7WiR5ErZl3

Name:              corey2-mirror2
State:             ACTIVE
Tables present:    LIVE
Open count:        0
Event number:      1
Major, minor:      253, 4
Number of targets: 1
UUID: LVM-caAVDd2dPhYq3rRmtemqUvgAt90oxO1vC3HFk9CLB9ELCxhSZBOf1lkkE11Rdz25

Name:              corey3-mirror3_mimage_1
State:             SUSPENDED
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 10
Number of targets: 1
UUID: LVM-vCWvB54l0xEiiQuUH1anc8UJwd1KeH2RCPDWgbqT9UuFK4p25MUvzxtMCyqNu3uF

Name:              corey3-mirror3_mimage_0
State:             SUSPENDED
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 9
Number of targets: 1
UUID: LVM-vCWvB54l0xEiiQuUH1anc8UJwd1KeH2RJELilE98AuY6yV853GNVoL1XNmkDpTaK

Name:              VolGroup00-LogVol01
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 1
Number of targets: 1
UUID: LVM-TCx5xJ7FuRhXzJ4g7CvPsw2AhhFBNLQUvNlDc7SvClgdBMh2WD6TraPFgzjSVMRp

Name:              corey3-mirror3
State:             SUSPENDED
Tables present:    LIVE & INACTIVE
Open count:        0
Event number:      0
Major, minor:      253, 12
Number of targets: 1
UUID: LVM-vCWvB54l0xEiiQuUH1anc8UJwd1KeH2RrVxFHPheVHLeYGToOyHqciYLB1QRho7o

Name:              VolGroup00-LogVol00
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 0
Number of targets: 1
UUID: LVM-TCx5xJ7FuRhXzJ4g7CvPsw2AhhFBNLQUrfFjpdEgWnBeJx2UpyC5Mr6XpgXdHWCh

Name:              corey1-mirror1_mlog
State:             ACTIVE
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      253, 5
Number of targets: 1
UUID: LVM-0QLi8UKjRguEwy4KI1k9KriBg4lyb2j1hCJh1EVAOxhYcsIkh5Fwap1yOkxFeidY

[root@link-08 ~]# dmsetup status
corey1-mirror1_mimage_1: 0 20971520 linear
corey1-mirror1: 0 20971520 mirror 2 253:6 253:7 20480/20480 1 AA 3
clustered_disk 253:5 A
corey1-mirror1_mimage_0: 0 20971520 linear
corey2-mirror2_mimage_1: 0 20971520 linear
corey2-mirror2_mimage_0: 0 20971520 linear
corey2-mirror2: 0 20971520 mirror 2 253:2 253:3 20480/20480 1 AA 1 clustered_core
corey3-mirror3_mimage_1: 0 20971520 linear
corey3-mirror3_mimage_0: 0 20971520 linear
VolGroup00-LogVol01: 0 4063232 linear
corey3-mirror3: 0 20971520 mirror 2 253:9 253:10 804/20480 1 AA 1 clustered_core
VolGroup00-LogVol00: 0 151912448 linear
corey1-mirror1_mlog: 0 8192 linear




Version-Release number of selected component (if applicable):
2.6.9-55.ELlargesmp
cmirror-kernel-2.6.9-32.0
lvm2-cluster-2.02.21-7.el4
Comment 1 Corey Marthaler 2007-06-11 14:47:12 EDT
This issue it easily reproduced.
Comment 2 Corey Marthaler 2007-06-20 13:00:26 EDT
This issue appears fixed in the lastest kernel mods that Jon gave me
(2.6.9-55.8.ELsmp).
Comment 3 Jonathan Earl Brassow 2007-09-28 11:38:20 EDT
Also, try again with cmirror >= 9/28/07

assigned -> modified
Comment 4 Corey Marthaler 2007-11-08 11:52:09 EST
Was not able to reproduce this, marking verified.
Comment 6 Alasdair Kergon 2010-04-27 11:01:29 EDT
Assuming this VERIFIED fix got released.  Closing.
Reopen if it's not yet resolved.

Note You need to log in before you can comment on or make changes to this bug.