Bug 782156

Summary: cmirror create deadlock - 'clogd: cpg_initialize failed: Cannot join cluster'
Product: Red Hat Enterprise Linux 5 Reporter: Nenad Peric <nperic>
Component: lvm2-clusterAssignee: LVM and device-mapper development team <lvm-team>
Status: CLOSED DUPLICATE QA Contact: Cluster QE <mspqa-list>
Severity: high Docs Contact:
Priority: high    
Version: 5.8CC: agk, ccaulfie, cmarthal, dwysocha, heinzm, jbrassow, mbroz, prajnoha, prockai, thornber, zkabelac
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 767002 Environment:
Last Closed: 2012-01-16 17:00:05 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 767002    
Bug Blocks:    

Description Nenad Peric 2012-01-16 16:54:02 UTC 
+++ This bug was initially created as a clone of Bug #767002 +++

Description of problem:
The creation attempt of multiple cmirror eventually results in a deadlock.


SCENARIO - [many_mirrors]
Recreating VG and PVs to increase metadata size
  Writing physical volume data to disk "/dev/sdd1"
  Writing physical volume data to disk "/dev/sdd2"
  Writing physical volume data to disk "/dev/sde1"
  Writing physical volume data to disk "/dev/sde2"
  Writing physical volume data to disk "/dev/sdf1"
  Writing physical volume data to disk "/dev/sdf2"
  Writing physical volume data to disk "/dev/sdg1"
  Writing physical volume data to disk "/dev/sdg2"
  Writing physical volume data to disk "/dev/sdh1"
  Writing physical volume data to disk "/dev/sdh2"
Making 200 mirrors...
1 taft-04: lvcreate -m 1 -n 200_1 -L 25M --nosync mirror_sanity
  WARNING: New mirror won't be synchronised. Don't read what you didn't write!
2 taft-04: lvcreate -m 1 -n 200_2 -L 25M --nosync mirror_sanity
  WARNING: New mirror won't be synchronised. Don't read what you didn't write!

[...]

129 taft-02: lvcreate -m 1 -n 200_129 -L 25M --nosync mirror_sanity
  WARNING: New mirror won't be synchronised. Don't read what you didn't write!
130 taft-03: lvcreate -m 1 -n 200_130 -L 25M --nosync mirror_sanity
  WARNING: New mirror won't be synchronised. Don't read what you didn't write!

[DEADLOCK]

Dec 12 13:38:38 taft-01 qarshd[18232]: Running cmdline: lvcreate -m 1 -n 500_122 -L 25M --nosync mirror_sanity
Dec 12 13:38:43 taft-01 clogd[6351]: cpg_initialize failed:  Cannot join cluster
Dec 12 13:38:43 taft-01 clogd[6351]: clog_resume:  Failed to create cluster CPG
Dec 12 13:38:43 taft-01 lvm[6597]: Monitoring mirror device mirror_sanity-500_122 for events.
Dec 12 13:38:48 taft-01 clogd[6351]: cpg_initialize failed:  Cannot join cluster
Dec 12 13:38:48 taft-01 clogd[6351]: clog_resume:  Failed to create cluster CPG
Dec 12 13:38:48 taft-01 lvm[6597]: Monitoring mirror device mirror_sanity-500_123 for events.
Dec 12 13:38:48 taft-01 qarshd[18367]: Running cmdline: lvcreate -m 1 -n 500_124 -L 25M --nosync mirror_sanity
Dec 12 13:38:53 taft-01 clogd[6351]: cpg_initialize failed:  Cannot join cluster
Dec 12 13:38:53 taft-01 clogd[6351]: clog_resume:  Failed to create cluster CPG
Dec 12 13:38:53 taft-01 lvm[6597]: Monitoring mirror device mirror_sanity-500_124 for events.
Dec 12 13:38:53 taft-01 qarshd[18435]: Running cmdline: lvcreate -m 1 -n 500_125 -L 25M --nosync mirror_sanity
Dec 12 13:38:59 taft-01 clogd[6351]: cpg_initialize failed:  Cannot join cluster
Dec 12 13:38:59 taft-01 clogd[6351]: clog_resume:  Failed to create cluster CPG
Dec 12 13:38:59 taft-01 lvm[6597]: Monitoring mirror device mirror_sanity-500_125 for events.
Dec 12 13:39:04 taft-01 clogd[6351]: cpg_initialize failed:  Cannot join cluster
Dec 12 13:39:04 taft-01 clogd[6351]: clog_resume:  Failed to create cluster CPG
Dec 12 13:39:04 taft-01 lvm[6597]: Monitoring mirror device mirror_sanity-500_126 for events.


Version-Release number of selected component (if applicable):
2.6.18-274.el5

lvm2-2.02.88-5.el5    BUILT: Fri Dec  2 12:25:45 CST 2011
lvm2-cluster-2.02.88-5.el5    BUILT: Fri Dec  2 12:48:37 CST 2011
device-mapper-1.02.67-2.el5    BUILT: Mon Oct 17 08:31:56 CDT 2011
device-mapper-event-1.02.67-2.el5    BUILT: Mon Oct 17 08:31:56 CDT 2011
cmirror-1.1.39-14.el5    BUILT: Wed Nov  2 17:25:33 CDT 2011
kmod-cmirror-0.1.22-3.el5    BUILT: Tue Dec 22 13:39:47 CST 2009

--- Additional comment from pm-rhel on 2012-01-09 09:51:23 EST ---

This request was evaluated by Red Hat Product Management for inclusion in Red Hat Enterprise Linux 5.8 and Red Hat does not plan to fix this issue the currently developed update.

Contact your manager or support representative in case you need to escalate this bug.
Comment 1 Nenad Peric 2012-01-16 17:00:05 UTC 
Tested on rhel5.8 with:

*** This bug has been marked as a duplicate of bug 767002 ***