Bug 423471 - lvs stuck locking local volume group
Summary: lvs stuck locking local volume group
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Cluster Suite
Classification: Retired
Component: lvm2-cluster
Version: 4
Hardware: All
OS: Linux
low
low
Target Milestone: ---
Assignee: LVM and device-mapper development team
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2007-12-13 14:50 UTC by Nate Straz
Modified: 2010-05-14 22:57 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-05-14 22:57:53 UTC
Embargoed:


Attachments (Terms of Use)

Description Nate Straz 2007-12-13 14:50:27 UTC
Description of problem:

While running helter_skelter (cluster mirror failure scenarios), an lvs
command hung and stopped the test suite.  New lvs commands do execute
without hanging.

Version-Release number of selected component (if applicable):
lvm2-2.02.27-2.el4_6.1
lvm2-cluster-2.02.27-2.el4_6.1
cman-1.0.17-0


How reproducible:
Unknown

Actual results:

Output from lvm_backtraces.pl:

tank-01:
[root@tank-01 ~]# /tmp/lvm_backtraces.pl 
Backtrace for lvs (12589):
#1  0x0030d673 in __read_nocancel () from /lib/tls/libpthread.so.0
#2  0x080a1190 in _lock_for_cluster (cmd=51 '3', flags=Variable "flags" is not
available.)    at locking/cluster_locking.c:115
#3  0x080a15f6 in _lock_resource (cmd=0x88bfa70, resource=Variable "resource" is
not available.)    at locking/cluster_locking.c:415
#4  0x0808bd2a in _lock_vol (cmd=0x88bfa70, resource=0xfefef720 "VolGroup00",  
  flags=33) at locking/locking.c:321
        #######################################
        # _lock_vol flags = LCK_VG | LCK_HOLD | LCK_READ
        #######################################
#5  0x0808bf51 in lock_vol (cmd=0x88bfa70, vol=0x88d7758 "VolGroup00",    
flags=33) at locking/locking.c:357
#6  0x08067f52 in process_each_lv (cmd=0x88bfa70, argc=0, argv=0xfeff13e8,    
lock_type=33, handle=0x88d8e70, process_single=0x8064e5e <_lvs_single>)    at
toollib.c:346
#7  0x0806574e in _report (cmd=0x88bfa70, argc=0, argv=0xfeff13e8,    
report_type=LVS) at reporter.c:322
#8  0x0805b571 in lvm_run_command (cmd=0x88bfa70, argc=0, argv=0xfeff13e8)    at
lvmcmdline.c:927
#9  0x0805c1d9 in lvm2_main (argc=1, argv=0xfeff13e4, is_static=0)    at
lvmcmdline.c:1205

Dec 13 05:23:01 tank-01 lvm[31154]: No longer monitoring mirror device
helter_skelter-syncd_primary_core_2legs_1 for events 
Dec 13 05:23:03 tank-01 lvm[31154]: No longer monitoring mirror device
helter_skelter-syncd_primary_core_2legs_2 for events 
Dec 13 07:40:35 tank-01 sshd(pam_unix)[12622]: authentication failure; logname=
uid=0 euid=0 tty=ssh ruser= rhost=vpn-248-34.boston.redhat.com  user=root
Dec 13 07:40:40 tank-01 sshd(pam_unix)[12622]: session opened for user root by
(uid=0)
Dec 13 07:46:35 tank-01 dhclient: DHCPREQUEST on eth0 to 10.15.89.100 port 67
Dec 13 07:46:35 tank-01 dhclient: DHCPACK from 10.15.89.100
Dec 13 07:46:35 tank-01 dhclient: bound to 10.15.89.91 -- renewal in 9042 seconds.
Dec 13 08:05:23 tank-01 qarshd[12715]: Talking to peer 10.15.89.98:35399
Dec 13 08:05:23 tank-01 qarshd[12715]: Got a QP_RSTAT with path = /tmp 
Dec 13 08:05:23 tank-01 qarshd[12715]: Got a QP_RECVFILE with path =
/tmp/lvm_backtraces.pl, ifd = 5013, count = 3701, mode = 100755 
DLM lockspace 'clvmd'

Resource b0374c80 (parent 00000000). Name (len=64)
"vtL0PIU8TI3egMQVFRmJsGE4CAWsEZ0yRHAj2cpArpVycyP1ECSpSZIR83LpbKvm"  
Local Copy, Master is node [[tank-02]]
Granted Queue
000100f0 CR 24729 Master:     0001010c
Conversion Queue
Waiting Queue

Resource b0374bbc (parent 00000000). Name (len=64)
"vtL0PIU8TI3egMQVFRmJsGE4CAWsEZ0yCMVI917i9xKYJOPvTaPZXHuPocuokYm3"  
Master Copy
Granted Queue
00010215 CR 24729
00020234 CR 10579 Remote:   [[tank-03]] 000103ab
000200dc CR 6176 Remote:   [[tank-02]] 00010223
00010032 CR 8301 Remote:   [[tank-04]] 000200dd
Conversion Queue
Waiting Queue

Resource 9d6e62cc (parent 00000000). Name (len=12) "V_VolGroup00"  
Master Copy
Granted Queue
000201ba PR 12590
Conversion Queue
Waiting Queue

tank-02:
[root@tank-02 ~]# /tmp/lvm_backtraces.pl 
Dec 13 06:54:03 tank-02 dhclient: DHCPREQUEST on eth0 to 10.15.89.100 port 67
Dec 13 06:54:03 tank-02 dhclient: DHCPACK from 10.15.89.100
Dec 13 06:54:03 tank-02 dhclient: bound to 10.15.89.92 -- renewal in 10493 seconds.
Dec 13 07:40:35 tank-02 sshd(pam_unix)[18896]: authentication failure; logname=
uid=0 euid=0 tty=ssh ruser= rhost=vpn-248-34.boston.redhat.com  user=root
Dec 13 07:40:40 tank-02 sshd(pam_unix)[18896]: session opened for user root by
(uid=0)
Dec 13 08:03:10 tank-02 kernel: cdrom: open failed.
Dec 13 08:03:40 tank-02 last message repeated 2 times
Dec 13 08:05:24 tank-02 qarshd[18949]: Talking to peer 10.15.89.98:35400
Dec 13 08:05:24 tank-02 qarshd[18949]: Got a QP_RSTAT with path = /tmp 
Dec 13 08:05:24 tank-02 qarshd[18949]: Got a QP_RECVFILE with path =
/tmp/lvm_backtraces.pl, ifd = 5014, count = 3701, mode = 100755 
DLM lockspace 'clvmd'

Resource 72ca1c80 (parent 00000000). Name (len=64)
"vtL0PIU8TI3egMQVFRmJsGE4CAWsEZ0yRHAj2cpArpVycyP1ECSpSZIR83LpbKvm"  
Master Copy
Granted Queue
0001010c CR 24729 Remote:   [[tank-01]] 000100f0
00010152 CR 8301 Remote:   [[tank-04]] 00010179
00010340 CR 10579 Remote:   [[tank-03]] 0003004c
00010245 CR 6176
Conversion Queue
Waiting Queue

Resource 72ca1bbc (parent 00000000). Name (len=64)
"vtL0PIU8TI3egMQVFRmJsGE4CAWsEZ0yCMVI917i9xKYJOPvTaPZXHuPocuokYm3"  
Local Copy, Master is node [[tank-01]]
Granted Queue
00010223 CR 6176 Master:     000200dc
Conversion Queue
Waiting Queue


vgdisplay on tank-01:
[root@tank-01 ~]# vgdisplay
  /dev/sda1: open failed: No such device or address
  --- Volume group ---
  VG Name               helter_skelter
  System ID             
  Format                lvm2
  Metadata Areas        3
  Metadata Sequence No  236
  VG Access             read/write
  VG Status             resizable
  Clustered             yes
  Shared                no
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                3
  Act PV                3
  VG Size               644.37 GB
  PE Size               4.00 MB
  Total PE              164959
  Alloc PE / Size       400 / 1.56 GB
  Free  PE / Size       164559 / 642.81 GB
  VG UUID               vtL0PI-U8TI-3egM-QVFR-mJsG-E4CA-WsEZ0y
   
  --- Volume group ---
  VG Name               VolGroup00
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               37.16 GB
  PE Size               32.00 MB
  Total PE              1189
  Alloc PE / Size       1187 / 37.09 GB
  Free  PE / Size       2 / 64.00 MB
  VG UUID               qCFkt6-brNI-BEbA-m6X6-tda3-0Gvp-bSoNr1



Expected results:


Additional info:

Comment 1 Alasdair Kergon 2007-12-13 15:00:19 UTC
Those are old packages - you need to be using lvm2*-4 now not -2.  This might be
the problem we fixed.

Comment 2 Nate Straz 2007-12-13 15:12:18 UTC
This is a z-stream candidate package.  We do not have the option of running -4.
 FWIW, the z-stream package should have the same fixes as the -4 package.  If
not, please verify which change between -2 and -4 solves this issue.

Comment 3 Milan Broz 2007-12-13 15:31:17 UTC
z-stream package contains exactly the same patches originally from -4 release.



Comment 5 Milan Broz 2010-05-14 22:57:53 UTC
I think this was fixed already (as mentioned in comment #3).

Please reopen if it is still problem, thanks.


Note You need to log in before you can comment on or make changes to this bug.