Bug 1271299 - Global lock failed: error -218
Global lock failed: error -218
Status: CLOSED DUPLICATE of bug 1271291
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: lvm2 (Show other bugs)
7.2
x86_64 Linux
unspecified Severity medium
: rc
: ---
Assigned To: David Teigland
cluster-qe@redhat.com
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-13 10:57 EDT by Corey Marthaler
Modified: 2016-01-19 10:59 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-01-19 10:59:21 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Corey Marthaler 2015-10-13 10:57:11 EDT
Description of problem:
This is most likely related to (if not the exact same bug as) bug 1271291.

When exclusive locked LVs existed on VG, and those VGs are then removed, a VG can not be created on a different node, but can be on the same node that once held the exclusive lock.

 [root@harding-02 ~]# pvscan
  Skipping global lock: storage errors for sanlock leases
  PV /dev/sda2   VG rhel_harding-02   lvm2 [92.67 GiB / 60.00 MiB free]
  PV /dev/sdb1   VG rhel_harding-02   lvm2 [93.16 GiB / 0    free]
  PV /dev/sdc1   VG rhel_harding-02   lvm2 [93.16 GiB / 0    free]
  Total: 3 [278.98 GiB] / in use: 3 [278.98 GiB] / in no VG: 0 [0   ]

[root@harding-03 ~]# pvscan
  Skipping global lock: lockspace not found or started
  PV /dev/sda2   VG rhel_harding-03   lvm2 [92.67 GiB / 60.00 MiB free]
  PV /dev/sdb1   VG rhel_harding-03   lvm2 [93.16 GiB / 0    free]
  PV /dev/sdc1   VG rhel_harding-03   lvm2 [93.16 GiB / 0    free]
  Total: 3 [278.98 GiB] / in use: 3 [278.98 GiB] / in no VG: 0 [0   ]

[root@harding-02 ~]# dmsetup ls
mpathe  (253:7)
mpathe1 (253:17)
mpathd  (253:6)
mpathc  (253:5)
mpathb  (253:4)
mpathh1 (253:18)
mpathc1 (253:13)
mpatha  (253:3)
rhel_harding--02-home   (253:2)
mpathf1 (253:14)
mpatha1 (253:11)
rhel_harding--02-swap   (253:1)
rhel_harding--02-root   (253:0)
mpathd1 (253:16)
mpathh  (253:10)
mpathg  (253:8)
mpathg1 (253:12)
mpathb1 (253:15)
mpathf  (253:9)


[root@harding-03 ~]# dmsetup ls
mpathe  (253:9)
mpathe1 (253:17)
mpathd  (253:4)
mpathc  (253:5)
mpathb  (253:7)
mpathh1 (253:18)
mpathc1 (253:12)
mpatha  (253:6)
mpathf1 (253:16)
rhel_harding--03-home   (253:2)
mpatha1 (253:15)
mpathd1 (253:11)
rhel_harding--03-swap   (253:1)
rhel_harding--03-root   (253:0)
mpathh  (253:10)
mpathg  (253:3)
mpathg1 (253:13)
mpathb1 (253:14)
mpathf  (253:8)


[root@harding-02 ~]# systemctl status sanlock
â sanlock.service - Shared Storage Lease Manager
   Loaded: loaded (/usr/lib/systemd/system/sanlock.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2015-10-12 23:07:22 CDT; 15h ago
  Process: 3338 ExecStart=/lib/systemd/systemd-sanlock start (code=exited, status=0/SUCCESS)
 Main PID: 3356 (sanlock)
   CGroup: /system.slice/sanlock.service
           ââ3356 sanlock daemon -U sanlock -G sanlock
           ââ3357 sanlock daemon -U sanlock -G sanlock

Oct 13 14:12:12 harding-02.lab.msp.redhat.com sanlock[3356]: 2015-10-13 14:12:12-0500 55287 [3365]: r39 cmd_acquire 3,9,3369 invalid lockspace found -1 failed 0 name lvm_snapper_thinp
Oct 13 14:12:48 harding-02.lab.msp.redhat.com sanlock[3356]: 2015-10-13 14:12:48-0500 55323 [3364]: r40 cmd_acquire 3,9,3369 invalid lockspace found -1 failed 0 name lvm_snapper_thinp
Oct 13 14:12:50 harding-02.lab.msp.redhat.com sanlock[3356]: 2015-10-13 14:12:50-0500 55325 [3365]: r41 cmd_acquire 3,9,3369 invalid lockspace found -1 failed 0 name lvm_snapper_thinp
Oct 13 14:14:48 harding-02.lab.msp.redhat.com sanlock[3356]: 2015-10-13 14:14:48-0500 55443 [3364]: r42 cmd_acquire 3,9,3369 invalid lockspace found -1 failed 0 name lvm_snapper_thinp
Oct 13 14:15:56 harding-02.lab.msp.redhat.com sanlock[3356]: 2015-10-13 14:15:56-0500 55511 [3365]: r43 cmd_acquire 3,9,3369 invalid lockspace found -1 failed 0 name lvm_snapper_thinp
Oct 13 14:16:01 harding-02.lab.msp.redhat.com sanlock[3356]: 2015-10-13 14:16:01-0500 55516 [3364]: r44 cmd_acquire 3,9,3369 invalid lockspace found -1 failed 0 name lvm_snapper_thinp
Oct 13 14:35:37 harding-02.lab.msp.redhat.com sanlock[3356]: 2015-10-13 14:35:37-0500 56692 [3365]: r45 cmd_acquire 3,9,3369 invalid lockspace found -1 failed 0 name lvm_snapper_thinp
Oct 13 14:35:58 harding-02.lab.msp.redhat.com sanlock[3356]: 2015-10-13 14:35:58-0500 56712 [3364]: r46 cmd_acquire 3,9,3369 invalid lockspace found -1 failed 0 name lvm_snapper_thinp
Oct 13 14:39:51 harding-02.lab.msp.redhat.com sanlock[3356]: 2015-10-13 14:39:51-0500 56946 [3365]: r47 cmd_acquire 3,9,3369 invalid lockspace found -1 failed 0 name lvm_snapper_thinp
Oct 13 14:41:48 harding-02.lab.msp.redhat.com sanlock[3356]: 2015-10-13 14:41:48-0500 57063 [3364]: r48 cmd_acquire 3,9,3369 invalid lockspace found -1 failed 0 name lvm_snapper_thinp

[root@harding-03 ~]# systemctl status sanlock
â sanlock.service - Shared Storage Lease Manager
   Loaded: loaded (/usr/lib/systemd/system/sanlock.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2015-10-12 23:07:06 CDT; 15h ago
  Process: 3255 ExecStart=/lib/systemd/systemd-sanlock start (code=exited, status=0/SUCCESS)
 Main PID: 3273 (sanlock)
   CGroup: /system.slice/sanlock.service
           ââ3273 sanlock daemon -U sanlock -G sanlock
           ââ3274 sanlock daemon -U sanlock -G sanlock

Oct 12 23:07:05 harding-03.lab.msp.redhat.com systemd[1]: Starting Shared Storage Lease Manager...
Oct 12 23:07:06 harding-03.lab.msp.redhat.com systemd-sanlock[3255]: Starting sanlock: [  OK  ]
Oct 12 23:07:06 harding-03.lab.msp.redhat.com systemd[1]: Started Shared Storage Lease Manager.


[root@harding-02 ~]# systemctl status lvm2-lvmlockd
â lvm2-lvmlockd.service - LVM2 lock daemon
   Loaded: loaded (/usr/lib/systemd/system/lvm2-lvmlockd.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2015-10-12 23:07:23 CDT; 15h ago
     Docs: man:lvmlockd(8)
 Main PID: 3369 (lvmlockd)
   CGroup: /system.slice/lvm2-lvmlockd.service
           ââ3369 /usr/sbin/lvmlockd -f

Oct 13 14:16:01 harding-02.lab.msp.redhat.com lvmlockd[3369]: 1444763761 S lvm_snapper_thinp R GLLK lock_san acquire error -28
Oct 13 14:16:01 harding-02.lab.msp.redhat.com lvmlockd[3369]: 1444763761 S lvm_snapper_thinp R GLLK res_lock lm error -218
Oct 13 14:35:37 harding-02.lab.msp.redhat.com lvmlockd[3369]: 1444764937 S lvm_snapper_thinp R GLLK lock_san acquire error -28
Oct 13 14:35:37 harding-02.lab.msp.redhat.com lvmlockd[3369]: 1444764937 S lvm_snapper_thinp R GLLK res_lock lm error -218
Oct 13 14:35:58 harding-02.lab.msp.redhat.com lvmlockd[3369]: 1444764958 S lvm_snapper_thinp R GLLK lock_san acquire error -28
Oct 13 14:35:58 harding-02.lab.msp.redhat.com lvmlockd[3369]: 1444764958 S lvm_snapper_thinp R GLLK res_lock lm error -218
Oct 13 14:39:51 harding-02.lab.msp.redhat.com lvmlockd[3369]: 1444765191 S lvm_snapper_thinp R GLLK lock_san acquire error -28
Oct 13 14:39:51 harding-02.lab.msp.redhat.com lvmlockd[3369]: 1444765191 S lvm_snapper_thinp R GLLK res_lock lm error -218
Oct 13 14:41:48 harding-02.lab.msp.redhat.com lvmlockd[3369]: 1444765308 S lvm_snapper_thinp R GLLK lock_san acquire error -28
Oct 13 14:41:48 harding-02.lab.msp.redhat.com lvmlockd[3369]: 1444765308 S lvm_snapper_thinp R GLLK res_lock lm error -218


[root@harding-03 ~]# systemctl status lvm2-lvmlockd
â lvm2-lvmlockd.service - LVM2 lock daemon
   Loaded: loaded (/usr/lib/systemd/system/lvm2-lvmlockd.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2015-10-12 23:07:06 CDT; 15h ago
     Docs: man:lvmlockd(8)
 Main PID: 3286 (lvmlockd)
   CGroup: /system.slice/lvm2-lvmlockd.service
           ââ3286 /usr/sbin/lvmlockd -f

Oct 12 23:07:06 harding-03.lab.msp.redhat.com systemd[1]: Started LVM2 lock daemon.
Oct 12 23:07:06 harding-03.lab.msp.redhat.com systemd[1]: Starting LVM2 lock daemon...
Oct 12 23:07:06 harding-03.lab.msp.redhat.com lvmlockd[3286]: [D] creating /run/lvm/lvmlockd.socket
Oct 12 23:07:06 harding-03.lab.msp.redhat.com lvmlockd[3286]: 1444709226 lvmlockd started
Oct 12 23:56:20 harding-03.lab.msp.redhat.com lvmlockd[3286]: 1444712180 S lvm_snapper_thinp R VGLK res_update version on non-ex lock
Oct 12 23:56:22 harding-03.lab.msp.redhat.com lvmlockd[3286]: 1444712182 S lvm_snapper_thinp R VGLK res_update version on non-ex lock

[root@harding-02 ~]# vgcreate --shared global /dev/mapper/mpatha1
  WARNING: shared lock type "sanlock" and lvmlockd are Technology Preview.
  For more information on Technology Preview features, visit:
  https://access.redhat.com/support/offerings/techpreview/
  Global lock failed: error -218

[root@harding-03 ~]# vgcreate --shared global /dev/mapper/mpatha1
  WARNING: shared lock type "sanlock" and lvmlockd are Technology Preview.
  For more information on Technology Preview features, visit:
  https://access.redhat.com/support/offerings/techpreview/
  Enabling sanlock global lock
  Physical volume "/dev/mapper/mpatha1" successfully created
  Physical volume "/dev/mapper/mpatha1" successfully created
  Logical volume "lvmlock" created.
  Physical volume "/dev/mapper/mpatha1" successfully created
  Volume group "global" successfully created
  VG global starting sanlock lockspace
  Starting locking.  Waiting until locks are ready...


Version-Release number of selected component (if applicable):
3.10.0-306.el7.x86_64

lvm2-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
lvm2-libs-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
lvm2-cluster-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-1.02.107-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-libs-1.02.107-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-event-1.02.107-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-event-libs-1.02.107-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-persistent-data-0.5.5-1.el7    BUILT: Thu Aug 13 09:58:10 CDT 2015
cmirror-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
sanlock-3.2.4-1.el7    BUILT: Fri Jun 19 12:48:49 CDT 2015
sanlock-lib-3.2.4-1.el7    BUILT: Fri Jun 19 12:48:49 CDT 2015
lvm2-lockd-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
Comment 2 David Teigland 2016-01-19 10:59:21 EST

*** This bug has been marked as a duplicate of bug 1271291 ***

Note You need to log in before you can comment on or make changes to this bug.