Bug 1264143

Summary: issues when removing first "global lock" vg before other shared vgs
Product: Red Hat Enterprise Linux 7 Reporter: Corey Marthaler <cmarthal>
Component: lvm2Assignee: LVM and device-mapper development team <lvm-team>
lvm2 sub component: LVM lock daemon / lvmlockd QA Contact: cluster-qe <cluster-qe>
Status: CLOSED NOTABUG Docs Contact:
Severity: medium    
Priority: unspecified CC: agk, heinzm, jbrassow, prajnoha, teigland, zkabelac
Version: 7.2   
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-09-17 18:27:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Corey Marthaler 2015-09-17 15:22:30 UTC 
Description of problem:
# setting up first "global lock" dummy vg for lvmlockd...
vgcreate --shared global /dev/mapper/mpatha1
harding-02: vgchange --lock-start global
harding-03: vgchange --lock-start global
  Skipping global lock: lockspace not found or started


# creating other shared VGs...
harding-02: pvcreate /dev/mapper/mpathb1 /dev/mapper/mpathc1 /dev/mapper/mpathd1 /dev/mapper/mpathe1
harding-02: vgcreate --shared raid_sanity /dev/mapper/mpathb1 /dev/mapper/mpathc1 /dev/mapper/mpathd1 /dev/mapper/mpathe1
harding-02: vgchange --lock-start raid_sanity
harding-03: vgchange --lock-start raid_sanity


[root@harding-02 ~]# lvs -a -o +devices
  LV        VG           Attr       LSize    Devices               
  [lvmlock] global       -wi-ao---- 256.00m  /dev/mapper/mpatha1(0)
  [lvmlock] raid_sanity  -wi-ao---- 256.00m  /dev/mapper/mpathb1(0)

[root@harding-03 ~]# lvs -a -o +devices
  LV        VG           Attr       LSize    Devices               
  [lvmlock] global       -wi-ao---- 256.00m  /dev/mapper/mpathg1(0)
  [lvmlock] raid_sanity  -wi-ao---- 256.00m  /dev/mapper/mpathd1(0)

[root@harding-02 ~]# pvscan
  PV /dev/mapper/mpatha1   VG global        lvm2 [249.99 GiB / 249.74 GiB free]
  PV /dev/mapper/mpathb1   VG raid_sanity   lvm2 [249.99 GiB / 249.74 GiB free]
  PV /dev/mapper/mpathc1   VG raid_sanity   lvm2 [249.99 GiB / 249.99 GiB free]
  PV /dev/mapper/mpathd1   VG raid_sanity   lvm2 [249.99 GiB / 249.99 GiB free]
  PV /dev/mapper/mpathe1   VG raid_sanity   lvm2 [249.99 GiB / 249.99 GiB free]

[root@harding-03 ~]# pvscan
  PV /dev/mapper/mpathg1   VG global        lvm2 [249.99 GiB / 249.74 GiB free]
  PV /dev/mapper/mpathd1   VG raid_sanity   lvm2 [249.99 GiB / 249.74 GiB free]
  PV /dev/mapper/mpathc1   VG raid_sanity   lvm2 [249.99 GiB / 249.99 GiB free]
  PV /dev/mapper/mpatha1   VG raid_sanity   lvm2 [249.99 GiB / 249.99 GiB free]
  PV /dev/mapper/mpathb1   VG raid_sanity   lvm2 [249.99 GiB / 249.99 GiB free]

[root@harding-02 ~]# vgchange --lock-stop global
[root@harding-03 ~]# vgchange --lock-stop global

[root@harding-02 ~]# vgremove global
  Global lock failed: check that global lockspace is started
[root@harding-02 ~]# vgchange --lock-start global
  Skipping global lock: lockspace not found or started
  VG global starting sanlock lockspace
  Starting locking.  Waiting until locks are ready...
[root@harding-02 ~]# vgremove global
  VG global held the sanlock global lock, enable global lock in another VG.
  Volume group "global" successfully removed

[root@harding-02 ~]# vgs
  Skipping global lock: lockspace not found or started
  VG              #PV #LV #SN Attr   VSize   VFree  
  raid_sanity       4   0   0 wz--ns 999.97g 999.72g

[root@harding-03 ~]# vgs
  Skipping global lock: lockspace not found or started
  VG              #PV #LV #SN Attr   VSize   VFree  
  raid_sanity       4   0   0 wz--ns 999.97g 999.72g

[root@harding-02 ~]# vgchange --lock-stop raid_sanity
[root@harding-03 ~]# vgchange --lock-stop raid_sanity

[root@harding-03 ~]# vgremove raid_sanity
  Global lock failed: check that global lockspace is started

[root@harding-02 ~]# vgchange --lock-start raid_sanity
  Skipping global lock: lockspace not found or started
  VG raid_sanity starting sanlock lockspace
  Starting locking.  Waiting until locks are ready...

[root@harding-02 ~]# vgremove raid_sanity
  Global lock failed: check that global lockspace is started

[root@harding-02 ~]# pvremove -ff --config global/use_lvmlockd=0 /dev/mapper/mpath[abcde]1
  Labels on physical volume "/dev/mapper/mpatha1" successfully wiped
Really WIPE LABELS from physical volume "/dev/mapper/mpathb1" of volume group "raid_sanity" [y/n]? y
  WARNING: Wiping physical volume label from /dev/mapper/mpathb1 of volume group "raid_sanity"
  Can't open /dev/mapper/mpathb1 exclusively - not removing. Mounted filesystem?
Really WIPE LABELS from physical volume "/dev/mapper/mpathc1" of volume group "raid_sanity" [y/n]? y
  WARNING: Wiping physical volume label from /dev/mapper/mpathc1 of volume group "raid_sanity"
  Labels on physical volume "/dev/mapper/mpathc1" successfully wiped
Really WIPE LABELS from physical volume "/dev/mapper/mpathd1" of volume group "raid_sanity" [y/n]? y
  WARNING: Wiping physical volume label from /dev/mapper/mpathd1 of volume group "raid_sanity"
  Labels on physical volume "/dev/mapper/mpathd1" successfully wiped
Really WIPE LABELS from physical volume "/dev/mapper/mpathe1" of volume group "raid_sanity" [y/n]? y
  WARNING: Wiping physical volume label from /dev/mapper/mpathe1 of volume group "raid_sanity"
  Labels on physical volume "/dev/mapper/mpathe1" successfully wiped

[root@harding-02 ~]# pvscan
  Skipping global lock: lockspace not found or started
  WARNING: Device for PV c9aYVL-Lach-Qa5X-c2wj-Zif3-oVXf-hVo3G0 not found or rejected by a filter.
  WARNING: Device for PV QaUfXm-bZcE-xN9x-Nkgl-FzAZ-2StN-AynRFF not found or rejected by a filter.
  WARNING: Device for PV X0I9qI-ixpd-494W-eULc-ZfTW-18UY-3gFqOC not found or rejected by a filter.
  WARNING: Device for PV c9aYVL-Lach-Qa5X-c2wj-Zif3-oVXf-hVo3G0 not found or rejected by a filter.
  WARNING: Device for PV QaUfXm-bZcE-xN9x-Nkgl-FzAZ-2StN-AynRFF not found or rejected by a filter.
  WARNING: Device for PV X0I9qI-ixpd-494W-eULc-ZfTW-18UY-3gFqOC not found or rejected by a filter.
  PV /dev/mapper/mpathb1   VG raid_sanity       lvm2 [249.99 GiB / 249.74 GiB free]
  PV unknown device        VG raid_sanity       lvm2 [249.99 GiB / 249.99 GiB free]
  PV unknown device        VG raid_sanity       lvm2 [249.99 GiB / 249.99 GiB free]
  PV unknown device        VG raid_sanity       lvm2 [249.99 GiB / 249.99 GiB free]

[root@harding-02 ~]# dmsetup ls
raid_sanity-lvmlock     (253:19)


Version-Release number of selected component (if applicable):
3.10.0-306.el7.x86_64

lvm2-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
lvm2-libs-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
lvm2-cluster-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-1.02.107-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-libs-1.02.107-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-event-1.02.107-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-event-libs-1.02.107-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
device-mapper-persistent-data-0.5.5-1.el7    BUILT: Thu Aug 13 09:58:10 CDT 2015
cmirror-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015
sanlock-3.2.4-1.el7    BUILT: Fri Jun 19 12:48:49 CDT 2015
sanlock-lib-3.2.4-1.el7    BUILT: Fri Jun 19 12:48:49 CDT 2015
lvm2-lockd-2.02.130-2.el7    BUILT: Tue Sep 15 07:15:40 CDT 2015


How reproducible:
Everytime
Comment 2 Corey Marthaler 2015-09-17 18:27:20 UTC 
global locks can be reestablished using the following command:
'lvmlockctl --gl-enable vg'