Bug 2098182

Summary: Allow changing VG system ID with majority of PVs
Product: Red Hat Enterprise Linux 9 Reporter: David Teigland <teigland>
Component: lvm2Assignee: David Teigland <teigland>
lvm2 sub component: Command-line tools QA Contact: cluster-qe <cluster-qe>
Status: VERIFIED --- Docs Contact:
Severity: medium    
Priority: high CC: agk, cmarthal, heinzm, jbrassow, mcsontos, msnitzer, prajnoha, thornber, zkabelac
Version: 9.0Keywords: Triaged
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: lvm2-2.03.21-1.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2174927 (view as bug list) Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2174911, 2174927    

Description David Teigland 2022-06-17 13:55:54 UTC 
Description of problem:

It should be possible to do failover of a VG (using LVM-activate) when the VG is missing devices.  Currently, the vgchange command to change the system ID requires all the PVs to be present.  This will likely require the majority of PVs to be present.

This request comes from bug 2066156.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 1 David Teigland 2022-10-14 16:45:56 UTC 
A commit for this exists in devel branch
https://sourceware.org/git/?p=lvm2.git;a=shortlog;h=refs/heads/dev-dct-vgchange-majority

but it's not yet clear that it would be used, e.g. by LVM-activate.
Comment 2 David Teigland 2023-03-08 22:51:23 UTC 
in main:
https://sourceware.org/git/?p=lvm2.git;a=commit;h=aa85ed1784b2b8e731ac7bd05a8988bac30405f0

This is meant for use by the LVM-activate resource agent, so the best testing will be from pacemaker testing with LVM-activate.
Basic functional test:

# vgcreate test /dev/loop0 /dev/loop1 /dev/loop2
  Physical volume "/dev/loop0" successfully created.
  Physical volume "/dev/loop1" successfully created.
  Physical volume "/dev/loop2" successfully created.
  Volume group "test" successfully created with system ID null-04.lab.msp.redhat.com

# vgs -o+systemid
  VG           #PV #LV #SN Attr   VSize    VFree   System ID                 
  test           3   0   0 wz--n-  180.00m 180.00m null-04.lab.msp.redhat.com

# losetup -d /dev/loop1
# losetup -d /dev/loop2

# pvs
  WARNING: Couldn't find device with uuid jQBR3w-RJko-tCbd-7yoD-YUOe-8Gd6-yeXmFC.
  WARNING: Couldn't find device with uuid Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG.
  WARNING: VG test is missing PV jQBR3w-RJko-tCbd-7yoD-YUOe-8Gd6-yeXmFC (last written to /dev/loop1).
  WARNING: VG test is missing PV Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG (last written to /dev/loop2).
  PV         VG           Fmt  Attr PSize    PFree  
  /dev/loop0 test         lvm2 a--    60.00m  60.00m
  [unknown]  test         lvm2 a-m    60.00m  60.00m
  [unknown]  test         lvm2 a-m    60.00m  60.00m

# vgchange --systemid foo test
  WARNING: Couldn't find device with uuid jQBR3w-RJko-tCbd-7yoD-YUOe-8Gd6-yeXmFC.
  WARNING: Couldn't find device with uuid Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG.
  WARNING: VG test is missing PV jQBR3w-RJko-tCbd-7yoD-YUOe-8Gd6-yeXmFC (last written to /dev/loop1).
  WARNING: VG test is missing PV Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG (last written to /dev/loop2).
  Cannot change VG test while PVs are missing.
  See vgreduce --removemissing and vgextend --restoremissing.
  Cannot process volume group test

# vgchange --majoritypvs --systemid foo test
  WARNING: Couldn't find device with uuid jQBR3w-RJko-tCbd-7yoD-YUOe-8Gd6-yeXmFC.
  WARNING: Couldn't find device with uuid Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG.
  WARNING: VG test is missing PV jQBR3w-RJko-tCbd-7yoD-YUOe-8Gd6-yeXmFC (last written to /dev/loop1).
  WARNING: VG test is missing PV Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG (last written to /dev/loop2).
  WARNING: Couldn't find device with uuid jQBR3w-RJko-tCbd-7yoD-YUOe-8Gd6-yeXmFC.
  WARNING: Couldn't find device with uuid Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG.
  Cannot change system ID without the majority of PVs (found 1 of 3)

# losetup -f loop1

# pvs
  WARNING: Couldn't find device with uuid Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG.
  WARNING: VG test is missing PV Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG (last written to /dev/loop2).
  PV         VG           Fmt  Attr PSize    PFree  
  /dev/loop0 test         lvm2 a--    60.00m  60.00m
  /dev/loop1 test         lvm2 a--    60.00m  60.00m
  [unknown]  test         lvm2 a-m    60.00m  60.00m

# vgchange --systemid foo test
  WARNING: Couldn't find device with uuid Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG.
  WARNING: VG test is missing PV Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG (last written to /dev/loop2).
  Cannot change VG test while PVs are missing.
  See vgreduce --removemissing and vgextend --restoremissing.
  Cannot process volume group test

# vgchange --majoritypvs --systemid foo test
  WARNING: Couldn't find device with uuid Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG.
  WARNING: VG test is missing PV Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG (last written to /dev/loop2).
  WARNING: Couldn't find device with uuid Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG.
  WARNING: Requested system ID foo does not match local system ID null-04.lab.msp.redhat.com.
  WARNING: Volume group test might become inaccessible from this machine.
Set foreign system ID foo on volume group test? [y/n]: y
  Volume group "test" successfully changed

# vgs --foreign -o+systemid
  WARNING: VG test is missing PV Hy4IWb-45o7-WCaF-VXHM-vWHi-6Gs5-JOqUlG (last written to [unknown]).
  VG           #PV #LV #SN Attr   VSize    VFree   System ID
  test           3   0   0 wz-pn-  180.00m 180.00m foo
Comment 5 Corey Marthaler 2023-05-16 19:36:18 UTC 
Marking Verified:Tested with the latest rpms.

kernel-5.14.0-306.el9    BUILT: Sat Apr 29 05:45:15 PM CEST 2023
lvm2-2.03.21-1.el9    BUILT: Fri Apr 21 02:33:33 PM CEST 2023
lvm2-libs-2.03.21-1.el9    BUILT: Fri Apr 21 02:33:33 PM CEST 2023



SCENARIO - attempt_vg_sysid_change_wo_majoritypvs_flag:  Test that changing the VG system ID is *NOT* supported when devices are missing when not using the --majoritypvs flag (bug 2098182) 
Setting system_id_source to enable
adding entry to the devices file for /dev/sdd1
creating PV on grant-01.6a2m.lab.eng.bos.redhat.com using device /dev/sdd1
pvcreate --yes -ff   /dev/sdd1
  Physical volume "/dev/sdd1" successfully created.
adding entry to the devices file for /dev/sde1
creating PV on grant-01.6a2m.lab.eng.bos.redhat.com using device /dev/sde1
pvcreate --yes -ff   /dev/sde1
  Physical volume "/dev/sde1" successfully created.
adding entry to the devices file for /dev/sdg1
creating PV on grant-01.6a2m.lab.eng.bos.redhat.com using device /dev/sdg1
pvcreate --yes -ff   /dev/sdg1
  Physical volume "/dev/sdg1" successfully created.
creating VG on grant-01.6a2m.lab.eng.bos.redhat.com using PV(s) /dev/sdd1 /dev/sde1 /dev/sdg1
vgcreate    df /dev/sdd1 /dev/sde1 /dev/sdg1
  Volume group "df" successfully created with system ID grant-01.6a2m.lab.eng.bos.redhat.com
VG systemid:grant-01.6a2m.lab.eng.bos.redhat.com

Disabling device sde on grant-01.6a2m.lab.eng.bos.redhat.com
'echo offline > /sys/block/sde/device/state'
Re-scanning lvm PVs (pvscan --cache)
  pvscan[84563] /dev/sde1 excluded: device is too small (pv_min_size).

Attempt to generate a new VG systemid for df w/o the use of the --majoritypvs flags
vgchange --yes --systemid virt-000.cluster-qe.lab.eng.brq.redhat.com df
WARNING: Couldn't find device with uuid BOdrgc-w7hS-75PL-UlO5-ih1H-OSvp-69DBHn.
  WARNING: VG df is missing PV BOdrgc-w7hS-75PL-UlO5-ih1H-OSvp-69DBHn (last written to /dev/sde1).
  Cannot change VG df while PVs are missing.
  See vgreduce --removemissing and vgextend --restoremissing.
  Cannot process volume group df
VG systemid:grant-01.6a2m.lab.eng.bos.redhat.com

Enabling device sde on grant-01.6a2m.lab.eng.bos.redhat.com
'echo running > /sys/block/sde/device/state'
Re-scanning lvm PVs (pvscan --cache)
  pvscan[84581] PV /dev/sde1 online.

removing vg df from grant-01.6a2m.lab.eng.bos.redhat.com
  Volume group "df" successfully removed
removing pv /dev/sdd1 on grant-01.6a2m.lab.eng.bos.redhat.com
  Labels on physical volume "/dev/sdd1" successfully wiped.
removing entry from the devices file for /dev/sdd1
removing pv /dev/sde1 on grant-01.6a2m.lab.eng.bos.redhat.com
  Labels on physical volume "/dev/sde1" successfully wiped.
removing entry from the devices file for /dev/sde1
removing pv /dev/sdg1 on grant-01.6a2m.lab.eng.bos.redhat.com
  Labels on physical volume "/dev/sdg1" successfully wiped.
removing entry from the devices file for /dev/sdg1
Setting system_id_source to disable


SCENARIO - vg_sysid_change_w_majoritypvs_flag_less_than_disabled:  Test that changing the VG system ID *IS* supported when less than majority devices are missing and using the --majoritypvs flag (bug 2098182) 
Setting system_id_source to enable
adding entry to the devices file for /dev/sdd1
creating PV on grant-01.6a2m.lab.eng.bos.redhat.com using device /dev/sdd1
pvcreate --yes -ff   /dev/sdd1
  Physical volume "/dev/sdd1" successfully created.
adding entry to the devices file for /dev/sde1
creating PV on grant-01.6a2m.lab.eng.bos.redhat.com using device /dev/sde1
pvcreate --yes -ff   /dev/sde1
  Physical volume "/dev/sde1" successfully created.
adding entry to the devices file for /dev/sdg1
creating PV on grant-01.6a2m.lab.eng.bos.redhat.com using device /dev/sdg1
pvcreate --yes -ff   /dev/sdg1
  Physical volume "/dev/sdg1" successfully created.
creating VG on grant-01.6a2m.lab.eng.bos.redhat.com using PV(s) /dev/sdd1 /dev/sde1 /dev/sdg1
vgcreate    df /dev/sdd1 /dev/sde1 /dev/sdg1
  Volume group "df" successfully created with system ID grant-01.6a2m.lab.eng.bos.redhat.com
VG systemid:grant-01.6a2m.lab.eng.bos.redhat.com

Disabling device sde on grant-01.6a2m.lab.eng.bos.redhat.com
'echo offline > /sys/block/sde/device/state'
Re-scanning lvm PVs (pvscan --cache)
  pvscan[84681] /dev/sde1 excluded: device is too small (pv_min_size).

Attempt to generate a new VG systemid w/o confirmation and ensure a 'foreign system ID' warning is present for df
vgchange --majoritypvs --systemid virt-000.cluster-qe.lab.eng.brq.redhat.com df
WARNING: Couldn't find device with uuid YqjH7J-WoqV-yv5V-E6pd-LbVy-uU36-VTyDE3.
  WARNING: VG df is missing PV YqjH7J-WoqV-yv5V-E6pd-LbVy-uU36-VTyDE3 (last written to /dev/sde1).
  WARNING: Couldn't find device with uuid YqjH7J-WoqV-yv5V-E6pd-LbVy-uU36-VTyDE3.
  WARNING: Requested system ID virt-000.cluster-qe.lab.eng.brq.redhat.com does not match local system ID grant-01.6a2m.lab.eng.bos.redhat.com.
  WARNING: Volume group df might become inaccessible from this machine.
Set foreign system ID virt-000.cluster-qe.lab.eng.brq.redhat.com on volume group df? [y/n]: [n]
  Volume group df system ID not changed.

Generating a new VG systemid w/ minority devices missing using the --majoritypvs flag for df
vgchange --yes --majoritypvs --systemid virt-000.cluster-qe.lab.eng.brq.redhat.com df
Volume group "df" successfully changed
WARNING: Couldn't find device with uuid YqjH7J-WoqV-yv5V-E6pd-LbVy-uU36-VTyDE3.
  WARNING: VG df is missing PV YqjH7J-WoqV-yv5V-E6pd-LbVy-uU36-VTyDE3 (last written to /dev/sde1).
  WARNING: Couldn't find device with uuid YqjH7J-WoqV-yv5V-E6pd-LbVy-uU36-VTyDE3.
  WARNING: Requested system ID virt-000.cluster-qe.lab.eng.brq.redhat.com does not match local system ID grant-01.6a2m.lab.eng.bos.redhat.com.
  WARNING: Volume group df might become inaccessible from this machine.

Enabling device sde on grant-01.6a2m.lab.eng.bos.redhat.com
'echo running > /sys/block/sde/device/state'
Re-scanning lvm PVs (pvscan --cache)
  pvscan[84697] PV /dev/sde1 online.

Stealing the foreign VG back from virt-000.cluster-qe.lab.eng.brq.redhat.com to original node grant-01.6a2m.lab.eng.bos.redhat.com
vgchange --yes --config local/extra_system_ids=virt-000.cluster-qe.lab.eng.brq.redhat.com --systemid grant-01.6a2m.lab.eng.bos.redhat.com df
Volume group "df" successfully changed
VG systemid:grant-01.6a2m.lab.eng.bos.redhat.com

removing vg df from grant-01.6a2m.lab.eng.bos.redhat.com
  Volume group "df" successfully removed
removing pv /dev/sdd1 on grant-01.6a2m.lab.eng.bos.redhat.com
  Labels on physical volume "/dev/sdd1" successfully wiped.
removing entry from the devices file for /dev/sdd1
removing pv /dev/sde1 on grant-01.6a2m.lab.eng.bos.redhat.com
  Labels on physical volume "/dev/sde1" successfully wiped.
removing entry from the devices file for /dev/sde1
removing pv /dev/sdg1 on grant-01.6a2m.lab.eng.bos.redhat.com
  Labels on physical volume "/dev/sdg1" successfully wiped.
removing entry from the devices file for /dev/sdg1
Setting system_id_source to disable


SCENARIO - vg_sysid_change_w_majoritypvs_flag_more_than_disabled:  Test that changing the VG system ID is *NOT* supported when more than majority devices are missing and using the --majoritypvs flag (bug 2098182) 
Setting system_id_source to enable
adding entry to the devices file for /dev/sdd1
creating PV on grant-01.6a2m.lab.eng.bos.redhat.com using device /dev/sdd1
pvcreate --yes -ff   /dev/sdd1
  Physical volume "/dev/sdd1" successfully created.
adding entry to the devices file for /dev/sde1
creating PV on grant-01.6a2m.lab.eng.bos.redhat.com using device /dev/sde1
pvcreate --yes -ff   /dev/sde1
  Physical volume "/dev/sde1" successfully created.
adding entry to the devices file for /dev/sdg1
creating PV on grant-01.6a2m.lab.eng.bos.redhat.com using device /dev/sdg1
pvcreate --yes -ff   /dev/sdg1
  Physical volume "/dev/sdg1" successfully created.
creating VG on grant-01.6a2m.lab.eng.bos.redhat.com using PV(s) /dev/sdd1 /dev/sde1 /dev/sdg1
vgcreate    df /dev/sdd1 /dev/sde1 /dev/sdg1
  Volume group "df" successfully created with system ID grant-01.6a2m.lab.eng.bos.redhat.com
VG systemid:grant-01.6a2m.lab.eng.bos.redhat.com

Disabling device sde on grant-01.6a2m.lab.eng.bos.redhat.com
'echo offline > /sys/block/sde/device/state'
Disabling device sdg on grant-01.6a2m.lab.eng.bos.redhat.com
'echo offline > /sys/block/sdg/device/state'
Re-scanning lvm PVs (pvscan --cache)
  pvscan[84804] /dev/sde1 excluded: device is too small (pv_min_size).

Attempt to generate a new VG systemid w/ majority devices missing using the --majoritypvs flag for df
vgchange --yes --majoritypvs --systemid virt-000.cluster-qe.lab.eng.brq.redhat.com df
WARNING: Couldn't find device with uuid ok96jD-n1bS-XQgu-xTTG-6z09-Fhm3-JO7Iww.
  WARNING: Couldn't find device with uuid iHLRrI-8VHG-NoJA-H4Zc-XqiL-brpa-RTueEi.
  WARNING: VG df is missing PV ok96jD-n1bS-XQgu-xTTG-6z09-Fhm3-JO7Iww (last written to /dev/sde1).
  WARNING: VG df is missing PV iHLRrI-8VHG-NoJA-H4Zc-XqiL-brpa-RTueEi (last written to /dev/sdg1).
  WARNING: Couldn't find device with uuid ok96jD-n1bS-XQgu-xTTG-6z09-Fhm3-JO7Iww.
  WARNING: Couldn't find device with uuid iHLRrI-8VHG-NoJA-H4Zc-XqiL-brpa-RTueEi.
  Cannot change system ID without the majority of PVs (found 1 of 3)
VG systemid:grant-01.6a2m.lab.eng.bos.redhat.com

Enabling device sde on grant-01.6a2m.lab.eng.bos.redhat.com
'echo running > /sys/block/sde/device/state'
Enabling device sdg on grant-01.6a2m.lab.eng.bos.redhat.com
'echo running > /sys/block/sdg/device/state'
Re-scanning lvm PVs (pvscan --cache)
  pvscan[84825] PV /dev/sde1 online.

removing vg df from grant-01.6a2m.lab.eng.bos.redhat.com
  Volume group "df" successfully removed
removing pv /dev/sdd1 on grant-01.6a2m.lab.eng.bos.redhat.com
  Labels on physical volume "/dev/sdd1" successfully wiped.
removing entry from the devices file for /dev/sdd1
removing pv /dev/sde1 on grant-01.6a2m.lab.eng.bos.redhat.com
  Labels on physical volume "/dev/sde1" successfully wiped.
removing entry from the devices file for /dev/sde1
removing pv /dev/sdg1 on grant-01.6a2m.lab.eng.bos.redhat.com
  Labels on physical volume "/dev/sdg1" successfully wiped.
removing entry from the devices file for /dev/sdg1
Setting system_id_source to disable
Comment 9 Corey Marthaler 2023-05-26 16:36:55 UTC 
Fix verified in the latest build as well. Marking VERIFIED.

kernel-5.14.0-312.el9    BUILT: Thu May 11 08:04:19 PM CEST 2023
lvm2-2.03.21-2.el9    BUILT: Thu May 25 12:03:04 AM CEST 2023
lvm2-libs-2.03.21-2.el9    BUILT: Thu May 25 12:03:04 AM CEST 2023



SCENARIO - vg_sysid_change_w_majoritypvs_flag_more_than_disabled:  Test that changing the VG system ID is *NOT* supported when more than majority devices are missing and using the --majoritypvs flag (bug 2098182) 
Setting system_id_source to enable
adding entry to the devices file for /dev/sdd
creating PV on virt-521.cluster-qe.lab.eng.brq.redhat.com using device /dev/sdd
pvcreate --yes -ff   /dev/sdd
  Physical volume "/dev/sdd" successfully created.
adding entry to the devices file for /dev/sda
creating PV on virt-521.cluster-qe.lab.eng.brq.redhat.com using device /dev/sda
pvcreate --yes -ff   /dev/sda
  Physical volume "/dev/sda" successfully created.
adding entry to the devices file for /dev/sde
creating PV on virt-521.cluster-qe.lab.eng.brq.redhat.com using device /dev/sde
pvcreate --yes -ff   /dev/sde
  Physical volume "/dev/sde" successfully created.
creating VG on virt-521.cluster-qe.lab.eng.brq.redhat.com using PV(s) /dev/sde /dev/sdd /dev/sda
vgcreate    df /dev/sde /dev/sdd /dev/sda
  Volume group "df" successfully created with system ID virt-521.cluster-qe.lab.eng.brq.redhat.com
VG systemid:virt-521.cluster-qe.lab.eng.brq.redhat.com

Disabling device sdd on virt-521.cluster-qe.lab.eng.brq.redhat.com
'echo offline > /sys/block/sdd/device/state'
Disabling device sda on virt-521.cluster-qe.lab.eng.brq.redhat.com
'echo offline > /sys/block/sda/device/state'
Re-scanning lvm PVs (pvscan --cache)
  pvscan[379883] /dev/sdd excluded: device is too small (pv_min_size).

Attempt to generate a new VG systemid w/ majority devices missing using the --majoritypvs flag for df
vgchange --yes --majoritypvs --systemid virt-000.cluster-qe.lab.eng.brq.redhat.com df
WARNING: Couldn't find device with uuid MZLf3l-v2mE-llZh-m6tN-dF1X-umdo-ZenvBe.
  WARNING: Couldn't find device with uuid AfRlMo-O7IT-Hm88-dWSz-9ZRu-mpbN-53KHns.
  WARNING: VG df is missing PV MZLf3l-v2mE-llZh-m6tN-dF1X-umdo-ZenvBe (last written to /dev/sdd).
  WARNING: VG df is missing PV AfRlMo-O7IT-Hm88-dWSz-9ZRu-mpbN-53KHns (last written to /dev/sda).
  WARNING: Couldn't find device with uuid MZLf3l-v2mE-llZh-m6tN-dF1X-umdo-ZenvBe.
  WARNING: Couldn't find device with uuid AfRlMo-O7IT-Hm88-dWSz-9ZRu-mpbN-53KHns.
  Cannot change system ID without the majority of PVs (found 1 of 3)
VG systemid:virt-521.cluster-qe.lab.eng.brq.redhat.com

Enabling device sdd on virt-521.cluster-qe.lab.eng.brq.redhat.com
'echo running > /sys/block/sdd/device/state'
Enabling device sda on virt-521.cluster-qe.lab.eng.brq.redhat.com
'echo running > /sys/block/sda/device/state'
Re-scanning lvm PVs (pvscan --cache)
  pvscan[380020] PV /dev/sdd online.

removing vg df from virt-521.cluster-qe.lab.eng.brq.redhat.com
  Volume group "df" successfully removed
removing pv /dev/sde on virt-521.cluster-qe.lab.eng.brq.redhat.com
  Labels on physical volume "/dev/sde" successfully wiped.
removing entry from the devices file for /dev/sde
removing pv /dev/sdd on virt-521.cluster-qe.lab.eng.brq.redhat.com
  Labels on physical volume "/dev/sdd" successfully wiped.
removing entry from the devices file for /dev/sdd
removing pv /dev/sda on virt-521.cluster-qe.lab.eng.brq.redhat.com
WARNING: Heartbeat did not respond at start of command.  Check btimed service on remote host
  Labels on physical volume "/dev/sda" successfully wiped.
removing entry from the devices file for /dev/sda
Setting system_id_source to disable


SCENARIO - vg_sysid_change_w_majoritypvs_flag_less_than_disabled:  Test that changing the VG system ID *IS* supported when less than majority devices are missing and using the --majoritypvs flag (bug 2098182) 
Setting system_id_source to enable
adding entry to the devices file for /dev/sdd
creating PV on virt-521.cluster-qe.lab.eng.brq.redhat.com using device /dev/sdd
pvcreate --yes -ff   /dev/sdd
  Physical volume "/dev/sdd" successfully created.
adding entry to the devices file for /dev/sde
creating PV on virt-521.cluster-qe.lab.eng.brq.redhat.com using device /dev/sde
pvcreate --yes -ff   /dev/sde
  Physical volume "/dev/sde" successfully created.
adding entry to the devices file for /dev/sda
creating PV on virt-521.cluster-qe.lab.eng.brq.redhat.com using device /dev/sda
pvcreate --yes -ff   /dev/sda
  Physical volume "/dev/sda" successfully created.
creating VG on virt-521.cluster-qe.lab.eng.brq.redhat.com using PV(s) /dev/sde /dev/sdd /dev/sda
vgcreate    df /dev/sde /dev/sdd /dev/sda
  Volume group "df" successfully created with system ID virt-521.cluster-qe.lab.eng.brq.redhat.com
VG systemid:virt-521.cluster-qe.lab.eng.brq.redhat.com

Disabling device sdd on virt-521.cluster-qe.lab.eng.brq.redhat.com
'echo offline > /sys/block/sdd/device/state'
Re-scanning lvm PVs (pvscan --cache)
  pvscan[380693] /dev/sdd excluded: device is too small (pv_min_size).

Attempt to generate a new VG systemid w/o confirmation and ensure a 'foreign system ID' warning is present for df
vgchange --majoritypvs --systemid virt-000.cluster-qe.lab.eng.brq.redhat.com df
WARNING: Couldn't find device with uuid 8AM1qQ-A5h2-5jOo-cZH3-AieZ-vQLQ-dTGn7i.
  WARNING: VG df is missing PV 8AM1qQ-A5h2-5jOo-cZH3-AieZ-vQLQ-dTGn7i (last written to /dev/sdd).
  WARNING: Couldn't find device with uuid 8AM1qQ-A5h2-5jOo-cZH3-AieZ-vQLQ-dTGn7i.
  WARNING: Requested system ID virt-000.cluster-qe.lab.eng.brq.redhat.com does not match local system ID virt-521.cluster-qe.lab.eng.brq.redhat.com.
  WARNING: Volume group df might become inaccessible from this machine.
Set foreign system ID virt-000.cluster-qe.lab.eng.brq.redhat.com on volume group df? [y/n]: [n]
  Volume group df system ID not changed.

Generating a new VG systemid w/ minority devices missing using the --majoritypvs flag for df
vgchange --yes --majoritypvs --systemid virt-000.cluster-qe.lab.eng.brq.redhat.com df
Volume group "df" successfully changed
WARNING: Couldn't find device with uuid 8AM1qQ-A5h2-5jOo-cZH3-AieZ-vQLQ-dTGn7i.
  WARNING: VG df is missing PV 8AM1qQ-A5h2-5jOo-cZH3-AieZ-vQLQ-dTGn7i (last written to /dev/sdd).
  WARNING: Couldn't find device with uuid 8AM1qQ-A5h2-5jOo-cZH3-AieZ-vQLQ-dTGn7i.
  WARNING: Requested system ID virt-000.cluster-qe.lab.eng.brq.redhat.com does not match local system ID virt-521.cluster-qe.lab.eng.brq.redhat.com.
  WARNING: Volume group df might become inaccessible from this machine.

Enabling device sdd on virt-521.cluster-qe.lab.eng.brq.redhat.com
'echo running > /sys/block/sdd/device/state'
Re-scanning lvm PVs (pvscan --cache)
  pvscan[380825] PV /dev/sdd online.

Stealing the foreign VG back from virt-000.cluster-qe.lab.eng.brq.redhat.com to original node virt-521.cluster-qe.lab.eng.brq.redhat.com
vgchange --yes --config local/extra_system_ids=virt-000.cluster-qe.lab.eng.brq.redhat.com --systemid virt-521.cluster-qe.lab.eng.brq.redhat.com df
Volume group "df" successfully changed
VG systemid:virt-521.cluster-qe.lab.eng.brq.redhat.com

removing vg df from virt-521.cluster-qe.lab.eng.brq.redhat.com
  Volume group "df" successfully removed
removing pv /dev/sde on virt-521.cluster-qe.lab.eng.brq.redhat.com
  Labels on physical volume "/dev/sde" successfully wiped.
removing entry from the devices file for /dev/sde
removing pv /dev/sdd on virt-521.cluster-qe.lab.eng.brq.redhat.com
  Labels on physical volume "/dev/sdd" successfully wiped.
removing entry from the devices file for /dev/sdd
removing pv /dev/sda on virt-521.cluster-qe.lab.eng.brq.redhat.com
  Labels on physical volume "/dev/sda" successfully wiped.
removing entry from the devices file for /dev/sda
Setting system_id_source to disable


SCENARIO - attempt_vg_sysid_change_wo_majoritypvs_flag:  Test that changing the VG system ID is *NOT* supported when devices are missing when not using the --majoritypvs flag (bug 2098182) 
Setting system_id_source to enable
adding entry to the devices file for /dev/sdd
creating PV on virt-521.cluster-qe.lab.eng.brq.redhat.com using device /dev/sdd
pvcreate --yes -ff   /dev/sdd
  Physical volume "/dev/sdd" successfully created.
adding entry to the devices file for /dev/sde
creating PV on virt-521.cluster-qe.lab.eng.brq.redhat.com using device /dev/sde
pvcreate --yes -ff   /dev/sde
  Physical volume "/dev/sde" successfully created.
adding entry to the devices file for /dev/sda
creating PV on virt-521.cluster-qe.lab.eng.brq.redhat.com using device /dev/sda
pvcreate --yes -ff   /dev/sda
  Physical volume "/dev/sda" successfully created.
creating VG on virt-521.cluster-qe.lab.eng.brq.redhat.com using PV(s) /dev/sde /dev/sdd /dev/sda
vgcreate    df /dev/sde /dev/sdd /dev/sda
  Volume group "df" successfully created with system ID virt-521.cluster-qe.lab.eng.brq.redhat.com
VG systemid:virt-521.cluster-qe.lab.eng.brq.redhat.com

Disabling device sdd on virt-521.cluster-qe.lab.eng.brq.redhat.com
'echo offline > /sys/block/sdd/device/state'
Re-scanning lvm PVs (pvscan --cache)
  pvscan[381562] /dev/sdd excluded: device is too small (pv_min_size).

Attempt to generate a new VG systemid for df w/o the use of the --majoritypvs flags
vgchange --yes --systemid virt-000.cluster-qe.lab.eng.brq.redhat.com df
WARNING: Couldn't find device with uuid 3nBEZy-ePLo-vmqm-niJi-CZp9-2gOh-LyKdDO.
  WARNING: VG df is missing PV 3nBEZy-ePLo-vmqm-niJi-CZp9-2gOh-LyKdDO (last written to /dev/sdd).
  Cannot change VG df while PVs are missing.
  See vgreduce --removemissing and vgextend --restoremissing.
  Cannot process volume group df
VG systemid:virt-521.cluster-qe.lab.eng.brq.redhat.com

Enabling device sdd on virt-521.cluster-qe.lab.eng.brq.redhat.com
'echo running > /sys/block/sdd/device/state'
Re-scanning lvm PVs (pvscan --cache)
  pvscan[381680] PV /dev/sdd online.

removing vg df from virt-521.cluster-qe.lab.eng.brq.redhat.com
  Volume group "df" successfully removed
removing pv /dev/sde on virt-521.cluster-qe.lab.eng.brq.redhat.com
  Labels on physical volume "/dev/sde" successfully wiped.
removing entry from the devices file for /dev/sde
removing pv /dev/sdd on virt-521.cluster-qe.lab.eng.brq.redhat.com
  Labels on physical volume "/dev/sdd" successfully wiped.
removing entry from the devices file for /dev/sdd
removing pv /dev/sda on virt-521.cluster-qe.lab.eng.brq.redhat.com
  Labels on physical volume "/dev/sda" successfully wiped.
removing entry from the devices file for /dev/sda
Setting system_id_source to disable