Bug 431785 - RHEL5 cmirror tracker: resync can get stuck
RHEL5 cmirror tracker: resync can get stuck
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: cmirror (Show other bugs)
5.2
All Linux
low Severity low
: rc
: ---
Assigned To: Jonathan Earl Brassow
Cluster QE
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-02-06 17:45 EST by Corey Marthaler
Modified: 2010-04-27 11:03 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-04-27 11:03:39 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Corey Marthaler 2008-02-06 17:45:46 EST
Description of problem:
This may be related to bz 431714. I've been able to reproduce a scenario where a
mirror attempting to resync gets stuck.

lvchange --resync -y mirror_sanity/resync_regular

[root@grant-02 ~]# lvs
  LV             VG            Attr   LSize  Origin Snap%  Move Log            
    Copy%  Convert
  LogVol00       VolGroup00    -wi-ao 72.44G
  LogVol01       VolGroup00    -wi-ao  1.94G
  resync_nosync  mirror_sanity mwi-a-  2.00G                   
resync_nosync_mlog  100.00
  resync_regular mirror_sanity mwi-a-  2.00G                   
resync_regular_mlog  10.55

# It never gets beyond 10.55 percent

# Grant-02:
Feb  6 15:57:41 grant-02 qarshd[11680]: Running cmdline: lvchange -ay
mirror_sanity/resync_nosync
Feb  6 15:57:41 grant-02 clogd[26975]: [zPCqDSfd] Cluster log created
Feb  6 15:57:41 grant-02 clogd[26975]: [zPCqDSfd] Non-master resume: bits pre-loaded
Feb  6 15:57:41 grant-02 lvm[3357]: Monitoring mirror device
mirror_sanity-resync_nosync for events
Feb  6 15:57:41 grant-02 clogd[26975]:   Region already in-sync: 0
Feb  6 15:57:41 grant-02 clogd[26975]:   Region already in-sync: 6
Feb  6 15:57:41 grant-02 xinetd[2789]: EXIT: qarsh status=0 pid=11680
duration=0(sec)
Feb  6 15:57:41 grant-02 clogd[26975]:   Region already in-sync: 7

# Grant-01 (mirror server):
Feb  6 15:57:36 grant-01 lvm[3425]: Monitoring mirror device
mirror_sanity-resync_regular for events
Feb  6 15:57:45 grant-01 lvm[3425]: No longer monitoring mirror device
mirror_sanity-resync_regular for events                                        
                                                             Feb  6 15:57:45
grant-01 clogd[29619]: [eVDBier3] clog_postsuspend: leaving CPG                
          Feb  6 15:57:45 grant-01 clogd[29619]: [eVDBier3] clog_postsuspend:
finalizing                            Feb  6 15:57:45 grant-01 clogd[29619]:
[eVDBier3] Cluster log removed                                     Feb  6
15:57:45 grant-01 lvm[3425]: No longer monitoring mirror device
mirror_sanity-resync_nosync for events                                         
                                                             Feb  6 15:57:45
grant-01 clogd[29619]: [zPCqDSfd] clog_postsuspend: leaving CPG                
          Feb  6 15:57:45 grant-01 clogd[29619]: [zPCqDSfd] clog_postsuspend:
finalizing                            Feb  6 15:57:45 grant-01 clogd[29619]:
[zPCqDSfd] Cluster log removed                                     Feb  6
15:57:49 grant-01 clogd[29619]: [eVDBier3] Cluster log created                 
                   Feb  6 15:57:49 grant-01 clogd[29619]: [eVDBier3] Master
resume: reading disk log                         Feb  6 15:57:49 grant-01
clogd[29619]: Header not valid                                                 
 Feb  6 15:57:49 grant-01 clogd[29619]:   magic     : 0  (expected: 4d695272)  
                           Feb  6 15:57:49 grant-01 clogd[29619]:   version   :
0                                                    Feb  6 15:57:49 grant-01
clogd[29619]:   nr_regions: 0                                                  
 Feb  6 15:57:49 grant-01 clogd[29619]: *** Invalid argument ***               
                           Feb  6 15:57:49 grant-01 [3425]: Monitoring mirror
device mirror_sanity-resync_regular for events         Feb  6 15:57:49 grant-01
clogd[29619]: [zPCqDSfd] Cluster log created                                   
 Feb  6 15:57:49 grant-01 clogd[29619]: [zPCqDSfd] Non-master resume: bits
pre-loaded                      Feb  6 15:57:49 grant-01 lvm[3425]: Monitoring
mirror device mirror_sanity-resync_nosync for events       Feb  6 15:57:49
grant-01 clogd[29619]:   Region already in-sync: 6                             
          Feb  6 15:57:49 grant-01 clogd[29619]:   Region already in-sync: 7   
                                    Feb  6 15:57:51 grant-01 clogd[29619]: []
Unable to send (null) to cluster: Invalid exchange              Feb  6 15:57:51
grant-01 clogd[29619]: Bad callback on local/4                                 
          Feb  6 15:57:56 grant-01 kernel: device-mapper: dm-log-clustered:
Timed out waiting for cluster log server [DM_CLOG_SET_REGION_SYNC]             
                                                                  Feb  6
15:57:56 grant-01 kernel: device-mapper: dm-log-clustered: Retrying request
[DM_CLOG_SET_REGION_SYNC]                                                      
                                                 Feb  6 15:57:56 grant-01
kernel: device-mapper: dm-log-clustered: [102282][0][0][0][0][0] [3]             


Grant-03:
Feb  6 15:57:57 grant-03 lvm[3369]: No longer monitoring mirror device
mirror_sanity-resync_regular for events
Feb  6 15:57:58 grant-03 clogd[1698]: [eVDBier3] clog_postsuspend: leaving CPG
Feb  6 15:57:58 grant-03 clogd[1698]: [eVDBier3] clog_postsuspend: finalizing
Feb  6 15:57:58 grant-03 clogd[1698]: [eVDBier3] Cluster log removed
Feb  6 15:57:58 grant-03 lvm[3369]: No longer monitoring mirror device
mirror_sanity-resync_nosync for events
Feb  6 15:57:58 grant-03 clogd[1698]: [zPCqDSfd] clog_postsuspend: leaving CPG
Feb  6 15:57:58 grant-03 clogd[1698]: [zPCqDSfd] clog_postsuspend: finalizing
Feb  6 15:57:58 grant-03 clogd[1698]: [zPCqDSfd] Cluster log removed
Feb  6 15:58:01 grant-03 clogd[1698]: [eVDBier3] Cluster log created
Feb  6 15:58:01 grant-03 clogd[1698]: [eVDBier3] Non-master resume: bits pre-loaded
Feb  6 15:58:01 grant-03 [3369]: Monitoring mirror device
mirror_sanity-resync_regular for events
Feb  6 15:58:01 grant-03 clogd[1698]: [zPCqDSfd] Cluster log created
Feb  6 15:58:01 grant-03 clogd[1698]: [zPCqDSfd] Master resume: reading disk log
Feb  6 15:58:01 grant-03 clogd[1698]: Header not valid
Feb  6 15:58:01 grant-03 clogd[1698]:   magic     : 0  (expected: 4d695272)
Feb  6 15:58:01 grant-03 clogd[1698]:   version   : 0
Feb  6 15:58:01 grant-03 clogd[1698]:   nr_regions: 0
Feb  6 15:58:01 grant-03 clogd[1698]: *** Invalid argument ***
Feb  6 15:58:01 grant-03 lvm[3369]: Monitoring mirror device
mirror_sanity-resync_nosync for events
Feb  6 15:58:02 grant-03 clogd[1698]:   Region already in-sync: 0
Feb  6 15:58:02 grant-03 clogd[1698]:   Region already in-sync: 6
Feb  6 15:58:02 grant-03 clogd[1698]:   Region already in-sync: 7
Feb  6 15:58:42 grant-03 xinetd[2794]: START: qarsh pid=18158 from=10.15.80.47
Feb  6 15:58:42 grant-03 qarshd[18158]: Talking to peer 10.15.80.47:56305
Feb  6 15:58:42 grant-03 qarshd[18158]: Running cmdline: lvs --noheadings -o
copy_percent mirror_sanity/resync_regular
Feb  6 15:58:42 grant-03 xinetd[2794]: EXIT: qarsh status=0 pid=18158
duration=0(sec)


Version-Release number of selected component (if applicable):
cmirror-1.1.11-1.el5
kmod-cmirror-0.1.5-2.el5
lvm2-2.02.32-1.el5
lvm2-cluster-2.02.32-1.el5
Comment 1 RHEL Product and Program Management 2008-02-06 17:57:09 EST
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 3 Corey Marthaler 2008-08-22 11:41:26 EDT
Mirror resyncing has long been verified to now work.
Comment 5 Alasdair Kergon 2010-04-27 11:03:39 EDT
Assuming this VERIFIED fix got released.  Closing.
Reopen if it's not yet resolved.

Note You need to log in before you can comment on or make changes to this bug.