Bug 2040456

Summary:	improve handling of duplicate vgids
Product:	Red Hat Enterprise Linux 9	Reporter:	David Teigland <teigland>
Component:	lvm2	Assignee:	David Teigland <teigland>
lvm2 sub component:	Command-line tools	QA Contact:	cluster-qe <cluster-qe>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	high
Priority:	high	CC:	agk, cmarthal, heinzm, jbrassow, mcsontos, msnitzer, prajnoha, thornber, zkabelac
Version:	9.0	Keywords:	Triaged
Target Milestone:	rc	Flags:	pm-rhel: mirror+
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	lvm2-2.03.16-1.el9	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	2125023 (view as bug list)		Environment:
Last Closed:	2022-11-15 11:20:43 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2125023

Description David Teigland 2022-01-13 17:43:35 UTC

Description of problem:

The approach to duplicate VGIDs has been that it is not possible or not allowed, so the behavior has been undefined. The actual result was unpredictable and/or broken, and generally unhelpful. LVM would not recognize the situation of duplicate vgids and would plow ahead in various undefined ways.

Improve this by recognizing the problem, displaying the VGs that exist, and printing a warning to fix the problem. This allows users to recognize what the problem is, and gives them a good chance of fixing it. Using LVM with duplicate VGIDs still remains invalid and largely undefined, so the bar is quite low on the behavior to expect in that state. But, the goal is to inform users of what has happened, and ideally allow lvm to work sufficiently well to run vgchange -u to fix the problem.

It's possible to create this condition without too much difficulty by cloning PVs, followed by an incomplete attempt at making the two VGs unique (vgrename and pvchange -u, but missing vgchange -u.)

Problem reported upstream: https://listman.redhat.com/archives/linux-lvm/2021-December/msg00003.html

# vgcreate test /dev/loop0
Physical volume "/dev/loop0" successfully created.
Volume group "test" successfully created

# dd if=/dev/loop0 of=/dev/loop1 bs=1M count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16556 s, 6.3 MB/s

# losetup -d /dev/loop0

# vgs
VG #PV #LV #SN Attr VSize VFree
test 1 0 0 wz--n- 60.00m 60.00m

# vgrename test test2
Volume group "test" successfully renamed to "test2"

# pvchange -u /dev/loop1
Physical volume "/dev/loop1" changed
1 physical volume changed / 0 physical volumes not changed

# losetup -f loop0

Now two different VGs have duplicate VGIDs.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual results:

Expected results:

Additional info:

Comment 1 David Teigland 2022-01-13 17:45:45 UTC

wrong component from failed menu selection

Comment 2 David Teigland 2022-01-13 17:53:46 UTC

several upstream commits related to this:
https://sourceware.org/git/?p=lvm2.git;a=commit;h=5e428d22d9647a294eba65852078947df5ade48f
https://sourceware.org/git/?p=lvm2.git;a=commit;h=cb798ee1c102aadde93965a894c5aa59d4e76e4a
https://sourceware.org/git/?p=lvm2.git;a=commit;h=18f451e09e943eb3d51f8394f6fe8967e1e9f565
https://sourceware.org/git/?p=lvm2.git;a=commit;h=0c80ea88477b3511562036748d3bf837bf7b3e8b

Follow the steps above, then:

# pvs
  WARNING: fix duplicate VGID MUH5vj-Uee4-Zv5y-LKOy-OoVw-oM3W-Vy2s2Y for VGs test2 and test (see vgchange -u).
  WARNING: lookup found duplicate VGID MUH5vjUee4Zv5yLKOyOoVwoM3WVy2s2Y for VGs test2 and test.
  WARNING: lookup found duplicate VGID MUH5vjUee4Zv5yLKOyOoVwoM3WVy2s2Y for VGs test2 and test.
  WARNING: lookup found duplicate VGID MUH5vjUee4Zv5yLKOyOoVwoM3WVy2s2Y for VGs test2 and test.
  WARNING: lookup found duplicate VGID MUH5vjUee4Zv5yLKOyOoVwoM3WVy2s2Y for VGs test2 and test.
  PV         VG           Fmt  Attr PSize    PFree  
  /dev/loop0 test         lvm2 a--    60.00m  60.00m
  /dev/loop1 test2        lvm2 a--    60.00m  60.00m

# vgs -o+uuid
  WARNING: fix duplicate VGID MUH5vj-Uee4-Zv5y-LKOy-OoVw-oM3W-Vy2s2Y for VGs test2 and test (see vgchange -u).
  WARNING: lookup found duplicate VGID MUH5vjUee4Zv5yLKOyOoVwoM3WVy2s2Y for VGs test2 and test.
  WARNING: lookup found duplicate VGID MUH5vjUee4Zv5yLKOyOoVwoM3WVy2s2Y for VGs test2 and test.
  WARNING: lookup found duplicate VGID MUH5vjUee4Zv5yLKOyOoVwoM3WVy2s2Y for VGs test2 and test.
  WARNING: lookup found duplicate VGID MUH5vjUee4Zv5yLKOyOoVwoM3WVy2s2Y for VGs test2 and test.
  VG           #PV #LV #SN Attr   VSize    VFree  VG UUID                               
  test           1   0   0 wz--n-   60.00m 60.00m MUH5vj-Uee4-Zv5y-LKOy-OoVw-oM3W-Vy2s2Y
  test2          1   0   0 wz--n-   60.00m 60.00m MUH5vj-Uee4-Zv5y-LKOy-OoVw-oM3W-Vy2s2Y

# vgchange -u test2
  Volume group "test2" successfully changed

# vgs -o+uuid
  VG           #PV #LV #SN Attr   VSize    VFree  VG UUID                               
  rhel_null-04   1   3   0 wz--n- <464.76g     0  84FXwS-b5do-BcUt-Eekf-MzYC-XWns-RaaT1J
  test           1   0   0 wz--n-   60.00m 60.00m MUH5vj-Uee4-Zv5y-LKOy-OoVw-oM3W-Vy2s2Y
  test2          1   0   0 wz--n-   60.00m 60.00m R9LYEk-43Wc-zpyr-6IZR-7EyR-unVq-EI2vmU

Comment 6 David Teigland 2022-02-16 17:38:41 UTC

Here's how I'd try to do an equivalent with scsi devs (not actually tried this).  The devices file should just be disabled for this test to make it simpler.

# vgcreate test /dev/sda

# dd if=/dev/sda of=/dev/sdb bs=1M count=1

# echo offline /dev/sda

# vgrename test test2

# pvchange -u /dev/sdb

# echo online /dev/sda

At this point there are two vgs (test on sda and test2 on sdb) with duplicate vgids.

without the fix:
- lvm commands will have mixed up output and errors
- lvm commands may or may not be able to fix the problem

with the fix:
- lvm commands should report warnings about duplicate vgs and display the two vgs properly
- vgchange -u <vgname> can be used to correct the problem by changing the vgid on either vg

Comment 7 Corey Marthaler 2022-07-21 18:50:48 UTC

Marking Verified:Tested in the latest rpms.

kernel-5.14.0-130.el9    BUILT: Fri Jul 15 07:31:56 AM CDT 2022
lvm2-2.03.16-2.el9    BUILT: Thu Jul 14 11:45:18 AM CDT 2022
lvm2-libs-2.03.16-2.el9    BUILT: Thu Jul 14 11:45:18 AM CDT 2022

SCENARIO - duplicate_vg_uuids_with_devicesfile_disabled:  Test how LVM reacts to PVs having the same pv and vg uuids while devicesfile is disabled (bug 2040456) 
adding entry to the devices file for /dev/sdn1
creating PV on hayes-03 using device /dev/sdn1
pvcreate --yes -ff   /dev/sdn1
  Physical volume "/dev/sdn1" successfully created.
creating VG on hayes-03 using PV(s) /dev/sdn1
vgcreate    df /dev/sdn1
  Volume group "df" successfully created

Disabling devicesfile on hayes-03
Setting use_devicesfile to disable
  PV /dev/sdn1   VG df              lvm2 [<1.82 TiB / <1.82 TiB free]
  Total: 1 [<1.82 TiB] / in use: 1 [<1.82 TiB] / in no VG: 0 [0   ]

Copying header from /dev/sdn1 to /dev/sdm1 on hayes-03
1+0 records in
1+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651065 s, 16.1 MB/s
Re-scanning lvm PVs (pvscan --cache)
  pvscan[4348] PV /dev/sdn1 online.

Disabling device sdn on hayes-03
'echo offline > /sys/block/sdn/device/state'
  VG #PV #LV #SN Attr   VSize  VFree 
  df   1   0   0 wz--n- <1.82t <1.82t
Renaming df to df_2
vgrename --yes df df_2
  Volume group "df" successfully renamed to "df_2"

Generating a new PV uuid for /dev/sdm1
pvchange -u /dev/sdm1

Enabling device sdn on hayes-03
'echo running > /sys/block/sdn/device/state'
WARNING: fix duplicate VGID cC9Or7-8fhy-V64N-spKL-y7QU-IhtC-KULXUH for VGs df and df_2 (see vgchange -u).
  WARNING: lookup found duplicate VGID cC9Or78fhyV64NspKLy7QUIhtCKULXUH for VGs df and df_2.
  WARNING: lookup found duplicate VGID cC9Or78fhyV64NspKLy7QUIhtCKULXUH for VGs df and df_2.
  WARNING: lookup found duplicate VGID cC9Or78fhyV64NspKLy7QUIhtCKULXUH for VGs df and df_2.
  WARNING: lookup found duplicate VGID cC9Or78fhyV64NspKLy7QUIhtCKULXUH for VGs df and df_2.

Generating a new VG uuid for df_2
vgchange -u df_2
Volume group "df_2" successfully changed
Generating a new VG uuid for df
vgchange -u df
Volume group "df" successfully changed

PV         VG   Fmt  Attr PSize  PFree 
  /dev/sdm1  df_2 lvm2 a--  <1.82t <1.82t
  /dev/sdn1  df   lvm2 a--  <1.82t <1.82t
removing vg df_2 from hayes-03
  Volume group "df_2" successfully removed
  Labels on physical volume "/dev/sdm1" successfully wiped.
  Volume group "df" successfully removed
Enabling devicesfile on hayes-03
Setting use_devicesfile to enable
removing pv /dev/sdn1 on hayes-03
  Labels on physical volume "/dev/sdn1" successfully wiped.
removing entry from the devices file for /dev/sdn1

Comment 11 errata-xmlrpc 2022-11-15 11:20:43 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (lvm2 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:8358