Bug 431621 - RHEL5 cmirror tracker: simultaneous creation can cause sync to get stuck
RHEL5 cmirror tracker: simultaneous creation can cause sync to get stuck
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: cmirror (Show other bugs)
5.2
All Linux
low Severity low
: rc
: ---
Assigned To: Jonathan Earl Brassow
Cluster QE
:
Depends On:
Blocks: 430797
  Show dependency treegraph
 
Reported: 2008-02-05 17:54 EST by Corey Marthaler
Modified: 2010-04-27 11:05 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-04-27 11:05:21 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Corey Marthaler 2008-02-05 17:54:37 EST
Description of problem:
I created a cmirror from all 4 nodes in the cluster at the same time. One of the
nodes got stuck while attempting to sync.

[root@taft-01 ~]# lvcreate -m 1 -n $(hostname) -L 2G taft
  Logical volume "taft-01" created

[root@taft-02 ~]# lvcreate -m 1 -n $(hostname) -L 2G taft
  Logical volume "taft-02" created

[root@taft-03 ~]# lvcreate -m 1 -n $(hostname) -L 2G taft
  Logical volume "taft-03" created

[root@taft-04 ~]# lvcreate -m 1 -n $(hostname) -L 2G taft
  Logical volume "taft-04" created


[root@taft-01 ~]# lvs -a -o +devices
  LV                 VG         Attr   LSize  Origin Snap%  Move Log         
Copy%  Convert Devices
  taft-01            taft       mwi-a-  2.00G                    taft-01_mlog 
36.33         taft-01_mimage_0(0),taft-01_mimage_)
  [taft-01_mimage_0] taft       Iwi-ao  2.00G                                  
             /dev/sdb1(0)
  [taft-01_mimage_1] taft       Iwi-ao  2.00G                                  
             /dev/sdc1(0)
  [taft-01_mlog]     taft       lwi-ao  4.00M                                  
             /dev/sdh1(0)

  taft-02            taft       mwi-a-  2.00G                    taft-02_mlog
100.00         taft-02_mimage_0(0),taft-02_mimage_)
  [taft-02_mimage_0] taft       iwi-ao  2.00G                                  
             /dev/sdh1(1)
  [taft-02_mimage_1] taft       iwi-ao  2.00G                                  
             /dev/sdb1(512)
  [taft-02_mlog]     taft       lwi-ao  4.00M                                  
             /dev/sdc1(514)

  taft-03            taft       mwi-a-  2.00G                    taft-03_mlog
100.00         taft-03_mimage_0(0),taft-03_mimage_)
  [taft-03_mimage_0] taft       iwi-ao  2.00G                                  
             /dev/sdf1(0)
  [taft-03_mimage_1] taft       iwi-ao  2.00G                                  
             /dev/sdg1(0)
  [taft-03_mlog]     taft       lwi-ao  4.00M                                  
             /dev/sdc1(513)

  taft-04            taft       mwi-a-  2.00G                    taft-04_mlog
100.00         taft-04_mimage_0(0),taft-04_mimage_)
  [taft-04_mimage_0] taft       iwi-ao  2.00G                                  
             /dev/sdd1(0)
  [taft-04_mimage_1] taft       iwi-ao  2.00G                                  
             /dev/sde1(0)
  [taft-04_mlog]     taft       lwi-ao  4.00M                                  
             /dev/sdc1(512)


Feb  5 16:39:34 taft-01 clogd[7471]: [peFjT2Ho] Cluster log created
Feb  5 16:39:34 taft-01 clogd[7471]: [peFjT2Ho] Unable to send
DM_CLOG_GET_SYNC_COUNT to cluster: Invalid exchange
Feb  5 16:39:34 taft-01 clogd[7471]: Bad callback on local/4
Feb  5 16:39:34 taft-01 kernel: device-mapper: dm-log-clustered: Server error
while processing request [DM_CLOG_GET_SYNC_COUNT]: -52
Feb  5 16:39:34 taft-01 clogd[7471]: Setting my cluster id: 1
Feb  5 16:39:34 taft-01 clogd[7471]: [peFjT2Ho] Non-master resume: bits pre-loaded
Feb  5 16:39:34 taft-01 [6781]: Monitoring mirror device taft-taft--01 for events
Feb  5 16:39:43 taft-01 clogd[7471]: [JNb0cmUL] Cluster log created
Feb  5 16:39:43 taft-01 clogd[7471]: [JNb0cmUL] Unable to send
DM_CLOG_GET_SYNC_COUNT to cluster: Invalid exchange
Feb  5 16:39:43 taft-01 clogd[7471]: Bad callback on local/4
Feb  5 16:39:43 taft-01 kernel: device-mapper: dm-log-clustered: Server error
while processing request [DM_CLOG_GET_SYNC_COUNT]: -52
Feb  5 16:39:43 taft-01 clogd[7471]: [JNb0cmUL] Master resume: reading disk log
Feb  5 16:39:43 taft-01 lvm[6781]: Monitoring mirror device taft-taft--04 for events
Feb  5 16:39:46 taft-01 clogd[7471]: [DOxLidpf] Cluster log created
Feb  5 16:39:46 taft-01 clogd[7471]: [DOxLidpf] Unable to send
DM_CLOG_GET_SYNC_COUNT to cluster: Invalid exchange
Feb  5 16:39:46 taft-01 clogd[7471]: Bad callback on local/4
Feb  5 16:39:46 taft-01 clogd[7471]: [DOxLidpf] Master resume: reading disk log
Feb  5 16:39:46 taft-01 kernel: device-mapper: dm-log-clustered: Server error
while processing request [DM_CLOG_GET_SYNC_COUNT]: -52
Feb  5 16:39:46 taft-01 lvm[6781]: Monitoring mirror device taft-taft--03 for events
Feb  5 16:39:50 taft-01 clogd[7471]: [Rk0MI94T] Cluster log created
Feb  5 16:39:50 taft-01 clogd[7471]: [Rk0MI94T] Unable to send
DM_CLOG_GET_SYNC_COUNT to cluster: Invalid exchange
Feb  5 16:39:50 taft-01 clogd[7471]: Bad callback on local/4
Feb  5 16:39:50 taft-01 clogd[7471]: [Rk0MI94T] Master resume: reading disk log
Feb  5 16:39:50 taft-01 kernel: device-mapper: dm-log-clustered: Server error
while processing request [DM_CLOG_GET_SYNC_COUNT]: -52
Feb  5 16:39:50 taft-01 lvm[6781]: Monitoring mirror device taft-taft--02 for events


Version-Release number of selected component (if applicable):
lvm2-2.02.32-1.el5
lvm2-cluster-2.02.32-1.el5
cmirror-1.1.9-1.el5
kmod-cmirror-0.1.5-2.el5
Comment 1 Corey Marthaler 2008-02-05 17:59:18 EST
Deactivating and then reactivating the volume group got the mirror to finally
sync properly.
Comment 2 RHEL Product and Program Management 2008-02-05 18:07:51 EST
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 4 Jonathan Earl Brassow 2008-02-13 13:31:12 EST
If you can recreate this, it will show more info when attempting to send the
request to the cluster.... would be nice to have.
Comment 5 Corey Marthaler 2008-02-29 10:27:40 EST
I haven't been able to reproduce this lately, marking verified.
cmirror-1.1.15-1.el5
kmod-cmirror-0.1.8-1.el5
Comment 7 Alasdair Kergon 2010-04-27 11:05:21 EDT
Assuming this VERIFIED fix got released.  Closing.
Reopen if it's not yet resolved.

Note You need to log in before you can comment on or make changes to this bug.