This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 979121 - lvm seems to report wrong precedence when having duplicate VGs/etc.
lvm seems to report wrong precedence when having duplicate VGs/etc.
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: lvm2 (Show other bugs)
7.3
All All
low Severity high
: pre-dev-freeze
: 7.3
Assigned To: David Teigland
cluster-qe@redhat.com
:
Depends On:
Blocks: 1313485 1295577
  Show dependency treegraph
 
Reported: 2013-06-27 12:29 EDT by Christoph Anton Mitterer
Modified: 2016-11-04 00:06 EDT (History)
11 users (show)

See Also:
Fixed In Version: lvm2-2.02.152-1.el7
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-11-04 00:06:29 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Christoph Anton Mitterer 2013-06-27 12:29:33 EDT
Hi.

When having duplicate VGs (i.e. same VG name but different UUID), like in the following case:
# vgdisplay 
  WARNING: Duplicate VG name vg_system: Existing mAn4TL-a3ey-WBDT-Q1Ul-i1Hi-kKLb-TmXMOq (created here) takes precedence over 08w1By-nKEW-oCKJ-SnV7-WF80-10EW-wRjGH0
  WARNING: Duplicate VG name vg_system: Existing mAn4TL-a3ey-WBDT-Q1Ul-i1Hi-kKLb-TmXMOq (created here) takes precedence over 08w1By-nKEW-oCKJ-SnV7-WF80-10EW-wRjGH0
  WARNING: Duplicate VG name vg_system: mAn4TL-a3ey-WBDT-Q1Ul-i1Hi-kKLb-TmXMOq (created here) takes precedence over 08w1By-nKEW-oCKJ-SnV7-WF80-10EW-wRjGH0
  --- Volume group ---
  VG Name               vg_system
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               238,47 GiB
  PE Size               4,00 MiB
  Total PE              61049
  Alloc PE / Size       61049 / 238,47 GiB
  Free  PE / Size       0 / 0   
  VG UUID               08w1By-nKEW-oCKJ-SnV7-WF80-10EW-wRjGH0
   
  WARNING: Duplicate VG name vg_system: mAn4TL-a3ey-WBDT-Q1Ul-i1Hi-kKLb-TmXMOq (created here) takes precedence over 08w1By-nKEW-oCKJ-SnV7-WF80-10EW-wRjGH0
  --- Volume group ---
  VG Name               vg_system
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               19,53 GiB
  PE Size               4,00 MiB
  Total PE              4999
  Alloc PE / Size       4999 / 19,53 GiB
  Free  PE / Size       0 / 0   
  VG UUID               mAn4TL-a3ey-WBDT-Q1Ul-i1Hi-kKLb-TmXMOq
   

The first one (08w1By-nKEW-oCKJ-SnV7-WF80-10EW-wRjGH0) was chronologically the frist one that became known to the system (and activated), the other one was later added as a loop device and then vgscan called (but not activated).

My observation is (and that seems logical), that the already activated takes precedence an /dev/vg_system continues to point to the first one.

However the warning seem to simply swap the UUIDs:
WARNING: Duplicate VG name vg_system: Existing mAn4TL-a3ey-WBDT-Q1Ul-i1Hi-kKLb-TmXMOq (created here) takes precedence over 08w1By-nKEW-oCKJ-SnV7-WF80-10EW-wRjGH0

Also why are the same warnings printed n duplicate times?

This problem may also exist with PVs and LVs...


Cheers,
Chris.
Comment 2 Christoph Anton Mitterer 2013-06-27 12:34:05 EDT
Another really serious problem seems to be, that already existing symlinks like /dev/disk/by-label/root are OVERWRITTEN with the newer devices/filesystem...
Comment 3 Alasdair Kergon 2013-06-27 12:47:59 EDT
LVM is not designed to support systems with two VGs that have the same name.  One of them should be renamed using vgrename and specifying its uuid.

Given that two such VGs are found, the tools have two choices.  Either they could refuse to do anything until the situation is resolved with a vgrename (or filters), or they could select one of the two VGs according some predefined rules.

They do the second.  Some people are advocating changing to the first because of the confusion it causes in situations like yours.

The rules for choosing between the two VGs have no access to information saying whether or not either of the VGs have active LVs and so this plays no part in the decision.  It would not be easy or cheap to add this and it would in turn add a different level of unpredictability to the decision.

We'll give this some thought.
Comment 4 Christoph Anton Mitterer 2013-06-27 17:42:07 EDT
Well first of all, support for different VGs with the same name should be improved, as this situation can very easily arise nowadays. Just consider you're working with VMs and directly mount their virtual disk in the host (which is quite common)... all of them usually have similar (a.k.a. the same names) like vg01 or vg_system or something like that.

It's even not that unlikely that you stumble across VGs/PVs/LVs with the same UUIDs, given that in VMs you simply often clone from a base image,... again,... support should be improved here... not sure if it's time to open a ticket for that.


I'm fine with the tools doing the second,.. but that means, IMHO, that they should generally _ALWAYS_ use those VG/LV/PVs, that were active (not known) chronologically first.
Because these are already used... and when you suddenly switch now, things may break.

I had the impression that this was more or less the case (at least my system continued to work)... but your text seems to imply that other rules control this?

Anyway... at least in case of stuff like /dev/disk/by-label/ it's definitely not working that the chronologically first one is used... but that's general problem not limited to LVM... is that udev's business.. cause I think one should become active there, too.


Nevertheless... this bug report was primarily about the warning message which seems to have mixed the two UUIDs?!
Comment 5 Alasdair Kergon 2013-06-27 17:50:51 EDT
The messages are accurate: they are nothing do with which VG has active LVs.

They tell you which VG would be processed if you used that particular VG name on the command line.  (When you give no VGs on the command line and the command expects to process a list of VGs, it generates that list internally then processes each item on it.)

But we will reconsider this area like you say: no need to open an additional bugzilla.
Comment 6 Christoph Anton Mitterer 2013-06-27 18:03:32 EDT
A now I understood :)

btw:
One idea to "handle" any collisions could be to use a special namespaces, which would have also been my solution for the genereal /dev/disk/by-[label|uuid] problem.

Like having:
/dev/vg-collisions/
respectively
/dev/disk/by-label/collisions/

and the content's would get a unique ID prefixed (which the system guarantees to be different) so one could have:
/dev/vg-collisions/1_vg_system/
/dev/vg-collisions/2_vg_system/
...
or
/dev/disk/by-label/collisions/1_root
/dev/disk/by-label/collisions/2_root
...

And a similar thing for the links in /dev/mapper/


What happens btw., when I create a VG called "null".. it can't set up /dev/null/ usually ;-)


Well it's up to you whether a new bug is used, I think so that there are three issues:
1) clarification of which duplicate the LVM tools use
I mean I'm not stupid but from that warning message I would not have realised that "what the LVM tools will use" is meant.

2) deciding whether the current choice of which duplicate is used (the first, the last, etc.) is best... or whether, as you said it's better to stop working at all until a rename has happened.

3) How one can handle collision of PV/LV/VG names at the /dev/ level.


Cheers,
Chris.
Comment 9 David Teigland 2015-10-15 13:53:14 EDT
The duplicate PV warning messages mentioned were recently fixed, improved, expanded, and made functional with lvmetad.  There is more work in progress in this area.  These changes might not make it back into RHEL6 unless lvm is rebased there.  They'll appear in the next RHEL7 update, so moving the bz there.
Comment 13 Mike McCune 2016-03-28 19:39:55 EDT
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune@redhat.com with any questions
Comment 15 Roman Bednář 2016-09-23 05:12:27 EDT
Duplicate PV/VG warning messages have been improved. 
Also lvmetad cache data are ignored in such situations to prevent other problems.

However, when trying to resolve the vg name duplicate I found that vgrename does not support --select|-S option, although it should.
vgrename can still be used by using uuid directly, but the select option is supposed to work and user gets a hint to use it when vgrename is used in a wrong way. See example below.

# pvs 
  WARNING: Not using lvmetad because duplicate PVs were found.
  WARNING: Use multipath or vgimportclone to resolve duplicate PVs?
  WARNING: After duplicates are resolved, run "pvscan --cache" to enable lvmetad.
  WARNING: PV 2qbsZ3-DWP7-3j7x-wFuV-tlat-Oe9z-iaKPh7 on /dev/sdb was already found on /dev/sda.
  WARNING: PV 2qbsZ3-DWP7-3j7x-wFuV-tlat-Oe9z-iaKPh7 prefers device /dev/sda because device was seen first.
  PV         VG            Fmt  Attr PSize   PFree  
  /dev/sda   vg_clone      lvm2 a--   40.00g  40.00g
  /dev/sdk   vg_clone      lvm2 a--  972.00m 972.00m
  ...

# systemctl is-active lvm2-lvmetad.service
active

======================================================

Here we try to resolve the conflict:

(Wrong arguments here on purpose, we get a hint to use select option)
# vgrename vg_clone vg_new
  WARNING: Not using lvmetad because duplicate PVs were found.
  WARNING: Use multipath or vgimportclone to resolve duplicate PVs?
  WARNING: After duplicates are resolved, run "pvscan --cache" to enable lvmetad.
  WARNING: PV 2qbsZ3-DWP7-3j7x-wFuV-tlat-Oe9z-iaKPh7 on /dev/sdb was already found on /dev/sda.
  WARNING: PV 2qbsZ3-DWP7-3j7x-wFuV-tlat-Oe9z-iaKPh7 prefers device /dev/sda because device was seen first.
  Multiple VGs found with the same name: skipping vg_clone
  Use --select vg_uuid=<uuid> in place of the VG name.

# vgs -o +vg_uuid
  WARNING: Not using lvmetad because duplicate PVs were found.
  WARNING: Use multipath or vgimportclone to resolve duplicate PVs?
  WARNING: After duplicates are resolved, run "pvscan --cache" to enable lvmetad.
  WARNING: PV 2qbsZ3-DWP7-3j7x-wFuV-tlat-Oe9z-iaKPh7 on /dev/sdb was already found on /dev/sda.
  WARNING: PV 2qbsZ3-DWP7-3j7x-wFuV-tlat-Oe9z-iaKPh7 prefers device /dev/sda because device was seen first.
  VG            #PV #LV #SN Attr   VSize   VFree   VG UUID                               
  rhel_virt-283   1   2   0 wz--n-   7.79g  40.00m TrH3Io-2XLs-omfO-J1K9-ZRoq-T2QG-TzwmuZ
  vg_clone        1   0   0 wz--n- 972.00m 972.00m 6Hwtsh-tkgf-oV2G-CqZs-0Kgr-QBgj-tL3EoN
  vg_clone        1   0   0 wz--n-  40.00g  40.00g xjv8tG-XDO0-PGZZ-4wGK-whMq-KOa3-5PUtMv

# vgrename --select vg_uuid=6Hwtsh-tkgf-oV2G-CqZs-0Kgr-QBgj-tL3EoN vg_clone vg_renamed
vgrename: unrecognized option '--select'
  Error during parsing of command line.



3.10.0-505.el7.x86_64

lvm2-2.02.165-2.el7    BUILT: Wed Sep 14 16:01:43 CEST 2016
lvm2-libs-2.02.165-2.el7    BUILT: Wed Sep 14 16:01:43 CEST 2016
lvm2-cluster-2.02.165-2.el7    BUILT: Wed Sep 14 16:01:43 CEST 2016
device-mapper-1.02.134-2.el7    BUILT: Wed Sep 14 16:01:43 CEST 2016
device-mapper-libs-1.02.134-2.el7    BUILT: Wed Sep 14 16:01:43 CEST 2016
device-mapper-event-1.02.134-2.el7    BUILT: Wed Sep 14 16:01:43 CEST 2016
device-mapper-event-libs-1.02.134-2.el7    BUILT: Wed Sep 14 16:01:43 CEST 2016
device-mapper-persistent-data-0.6.3-1.el7    BUILT: Fri Jul 22 12:29:13 CEST 2016
cmirror-2.02.165-2.el7    BUILT: Wed Sep 14 16:01:43 CEST 2016
Comment 16 Peter Rajnoha 2016-09-23 05:17:40 EDT
(In reply to Roman Bednář from comment #15)
> Here we try to resolve the conflict:
> 
> (Wrong arguments here on purpose, we get a hint to use select option)
> # vgrename vg_clone vg_new
>   WARNING: Not using lvmetad because duplicate PVs were found.
>   WARNING: Use multipath or vgimportclone to resolve duplicate PVs?
>   WARNING: After duplicates are resolved, run "pvscan --cache" to enable
> lvmetad.
>   WARNING: PV 2qbsZ3-DWP7-3j7x-wFuV-tlat-Oe9z-iaKPh7 on /dev/sdb was already
> found on /dev/sda.
>   WARNING: PV 2qbsZ3-DWP7-3j7x-wFuV-tlat-Oe9z-iaKPh7 prefers device /dev/sda
> because device was seen first.
>   Multiple VGs found with the same name: skipping vg_clone
>   Use --select vg_uuid=<uuid> in place of the VG name.
> 
...
> # vgrename --select vg_uuid=6Hwtsh-tkgf-oV2G-CqZs-0Kgr-QBgj-tL3EoN vg_clone
> vg_renamed
> vgrename: unrecognized option '--select'
>   Error during parsing of command line.

The vgrename uses process_each_vg so it uses common code which also includes support for "-S|--select". The only issue here is that vgrename doesn't have that argument registered with the vgrenamen command (which is easy to add). However, in this case, we need to be sure that the select always selects one VG, not more (...and the command should fail if the result of selection gives us more items - I'm not sure now if we have this check or not within the process_each_* code if used together with -S|--select, but if yes, it should be easy to switch vgrenamen into using -S now.)
Comment 17 Peter Rajnoha 2016-09-23 06:51:07 EDT
Separate BZ open to track the "vgrename -S|--select": bug #1378829.
Comment 18 Roman Bednář 2016-09-29 03:53:53 EDT
Marking verified with latest rpms.

3.10.0-505.el7.x86_64

lvm2-2.02.166-1.el7    BUILT: Wed Sep 28 09:26:52 CEST 2016
lvm2-libs-2.02.166-1.el7    BUILT: Wed Sep 28 09:26:52 CEST 2016
lvm2-cluster-2.02.166-1.el7    BUILT: Wed Sep 28 09:26:52 CEST 2016
device-mapper-1.02.135-1.el7    BUILT: Wed Sep 28 09:26:52 CEST 2016
device-mapper-libs-1.02.135-1.el7    BUILT: Wed Sep 28 09:26:52 CEST 2016
device-mapper-event-1.02.135-1.el7    BUILT: Wed Sep 28 09:26:52 CEST 2016
device-mapper-event-libs-1.02.135-1.el7    BUILT: Wed Sep 28 09:26:52 CEST 2016
device-mapper-persistent-data-0.6.3-1.el7    BUILT: Fri Jul 22 12:29:13 CEST 2016
cmirror-2.02.166-1.el7    BUILT: Wed Sep 28 09:26:52 CEST 2016
Comment 20 errata-xmlrpc 2016-11-04 00:06:29 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-1445.html

Note You need to log in before you can comment on or make changes to this bug.