This service will be undergoing maintenance at 20:00 UTC, 2017-04-03. It is expected to last about 30 minutes
Bug 131317 - clvmd mysteriously dies after too many lvcreates or lvremoves
clvmd mysteriously dies after too many lvcreates or lvremoves
Status: CLOSED CURRENTRELEASE
Product: Red Hat Cluster Suite
Classification: Red Hat
Component: gfs (Show other bugs)
4
i686 Linux
medium Severity medium
: ---
: ---
Assigned To: Christine Caulfield
GFS Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2004-08-30 17:32 EDT by Corey Marthaler
Modified: 2010-01-11 21:57 EST (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2004-10-27 16:45:37 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Corey Marthaler 2004-08-30 17:32:31 EDT
Description of problem:
Create a bunch of LVs and watch clvmd die silently. I can then
reproduce this everytime by removing the LVs and then creating them again.

[root@morph-02 root]# while true ; do lvcreate -L 0.009TB corey; done
  Rounding up size to full physical extent 9.22 GB
  Logical volume "lvol0" created
  Rounding up size to full physical extent 9.22 GB
  Logical volume "lvol1" created
  Rounding up size to full physical extent 9.22 GB
  Logical volume "lvol2" created
  . 
  .
  .
  Rounding up size to full physical extent 9.22 GB
  Logical volume "lvol59" created
  Rounding up size to full physical extent 9.22 GB
  Logical volume "lvol60" created
  Error reading data from clvmd: Connection reset by peer
  Can't get lock for corey
  connect() failed on local socket: Connection refused
  Locking type 2 initialisation failed.
  connect() failed on local socket: Connection refused
  Locking type 2 initialisation failed.


[root@morph-02 locking]# clvmd -d
[root@morph-02 locking]# echo $?
0


If I do:

'strace clvmd -d' I rarely ever see this problem and if I do it takes
a long time.

How reproducible:
Always
Comment 1 Christine Caulfield 2004-09-16 08:42:48 EDT
How long does this normally take to happen? I have managed to create
and remove around 2,000 LVs so far with no problems whatsoever.

how many nodes are in your cluster and what version of OS are they using?
Comment 2 Corey Marthaler 2004-09-16 11:12:59 EDT
It doesn't take long at all. I reproduced this bug everytime I tried 
this morning and hit it after only 15-28 iterations. 
 
There are 6 nodes in my cluster and are all running: 
Linux morph-01 2.6.8.1 #1 SMP Mon Sep 13 10:33:02 CDT 2004 i686 i686 
i386 GNU/Linux 
 
 
Comment 3 Alasdair Kergon 2004-09-16 16:09:56 EDT
Fixed one routine that claimed to free memory but didn't.
Comment 4 Corey Marthaler 2004-09-21 11:23:50 EDT
I still consistently see this issue. 
Comment 5 Christine Caulfield 2004-09-21 11:59:19 EDT
Is is possible to get any more debugging out of this ? (I can't make
it happen on any of my systems). A core dump/traceback would be ideal
but the output of "clvmd -d" might also be helpful if you can make it
happen with that enabled.
Comment 6 Christine Caulfield 2004-09-22 03:25:38 EDT
...or it might be more convenient to let me use your test system one
morning (my time)
Comment 7 Corey Marthaler 2004-09-22 17:08:32 EDT
"clvmd -d" doesn't report a single thing while deleting or creating 
lvs/pvs/vgs.  
 
I can leave my cluster idle tonight for you to test on.  
 
Here's what I do: 
 
[root@morph-02 root]# pvcreate /dev/sda 
  Physical volume "/dev/sda" successfully created 
[root@morph-02 root]# vgcreate corey /dev/sda 
  Volume group "corey" successfully created 
[root@morph-02 root]# vgchange -ay corey 
  0 logical volume(s) in volume group "corey" now active 
[root@morph-02 root]# while true ; do lvcreate -L 0.009TB corey; 
done 
  Rounding up size to full physical extent 9.22 GB 
  Logical volume "lvol0" created 
  Rounding up size to full physical extent 9.22 GB 
  Logical volume "lvol1" created 
  . 
  . 
  .    
  Rounding up size to full physical extent 9.22 GB 
  Logical volume "lvol28" created 
  connect() failed on local socket: Connection refused 
  Locking type 2 initialisation failed. 
  connect() failed on local socket: Connection refused 
  Locking type 2 initialisation failed. 
Comment 8 Christine Caulfield 2004-09-23 09:22:26 EDT
OK, some of the locking and thread handling was a bit dodgy. To be
honest I'm not sure whether what I've done has simply slowed it down
so that the bug doesn't happen any more or has actually also fixed it.
Comment 9 Corey Marthaler 2004-10-27 16:45:37 EDT
haven't seen this bug with extensive testing, since last fix.
Comment 10 Kiersten (Kerri) Anderson 2004-11-16 14:12:43 EST
Updating version to the right level in the defects.  Sorry for the storm.

Note You need to log in before you can comment on or make changes to this bug.