Bug 1730520 - if we have 2 vg's with the same name, after renaming one, it uses the other's PV's to get metadata!
Summary: if we have 2 vg's with the same name, after renaming one, it uses the other's...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: lvm2
Version: 7.6
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
: ---
Assignee: David Teigland
QA Contact: cluster-qe@redhat.com
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-17 03:28 UTC by nikhil kshirsagar
Modified: 2020-01-28 15:03 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-01-28 15:03:36 UTC
Target Upstream Version:


Attachments (Terms of Use)
vgscan verbose (1.96 MB, text/plain)
2019-07-17 03:28 UTC, nikhil kshirsagar
no flags Details
metadata from 1 of the 5 nodes (103.46 KB, application/gzip)
2019-07-17 03:35 UTC, nikhil kshirsagar
no flags Details
metadata from node jdb02 (160.15 KB, application/gzip)
2019-07-17 04:00 UTC, nikhil kshirsagar
no flags Details
verbose outputs of some lvm commands (183.59 KB, application/gzip)
2019-07-17 04:01 UTC, nikhil kshirsagar
no flags Details

Description nikhil kshirsagar 2019-07-17 03:28:42 UTC
Created attachment 1591239 [details]
vgscan verbose

Description of problem:
The customer opened a support case asking to remove a VG with a duplicate name. This probably happened because he attached a disk which had metadata (containing a vg with the same name as a vg currently present on this server)

So what they requested was, to know how to remove the duplicated manoeuvrevg1

we want to remove that extra VG(which is duplicated ) in this server.

VG           Attr   Ext   #PV #LV #SN VSize    VFree    VG UUID
manoeuvrevg1 wz-pn- 4.00m  15   1   0   <2.93t       0  g7ovuA-7dLz-xBH9-UMvX-rKkm-fXVv-rpC5Ch
manoeuvrevg1 wz-pn- 4.00m  15   1   0   <2.93t       0  XOrFC0-rdEC-VjqY-Rdcj-Kemn-I88r-V9hgJj
manoeuvrevg2 wz-pn- 4.00m  15   1   0   <2.93t       0  FtAV6z-TlT1-fpBU-bfsQ-3t0f-Eq8V-v2z64W


At this point use_lvmetad was set to 1 in lvm.conf , they do not use clvmd for this 5 node RAC cluster but told us that

"

These are manœuvre LUNs (shared LUNs), it’s supposed to be created / declared on ALL servers but mounted on only 1 server at once. 
This must not be mounted on multiple servers in the same time, for sure. 
This will be used to copy DB export from 1 server (A) to another (B) using below process : 
FS /manoeuvre1 and /manoeuvre2 are mounted on server A 
DB export is copied to FS /manoeuvre1 and /manoeuvre2 on server A 
FS /manoeuvre1 and /manoeuvre2 are unmounted on server A 
FS /manoeuvre1 and /manoeuvre2 are mounted on server B 
DB export is copied from FS /manoeuvre1 and /manoeuvre2 on server B 
FS /manoeuvre1 and /manoeuvre2 are unmounted on server B
"

Support then asked them to do this.

"

You have 2 VG with the same name:

VG           Attr   Ext   #PV #LV #SN VSize    VFree    VG UUID
manoeuvrevg1 wz-pn- 4.00m  15   0   0   <2.93t       0  g7ovuA-7dLz-xBH9-UMvX-rKkm-fXVv-rpC5Ch
manoeuvrevg1 wz--n- 4.00m  15   1   0   <2.93t       0  XOrFC0-rdEC-VjqY-Rdcj-Kemn-I88r-V9hgJj

Action Plan:

o Either remove the luns which are not used by active VG.

o Or, Change the Active VG name to something else like manoeuvrevg1_bkp with the help of uuid.

# vgrename uuid manoeuvrevg1_bkp

"

On hindsight it may have been better to do a vgremove here instead of a rename?

After this, what we ended up with was,

VG               #PV #LV #SN Attr   VSize    VFree    VG UUID
  applivg            3   3   0 wz--n- <599.99g <402.99g xVR2VB-tfeh-nZdH-BQhw-h6Xr-GrW1-So5rzG
  manoeuvrevg1      15   0   0 wz-pn-   <2.93t   <2.93t g7ovuA-7dLz-xBH9-UMvX-rKkm-fXVv-rpC5Ch
  manoeuvrevg1_bkp  15   1   0 wz--n-   <2.93t       0  XOrFC0-rdEC-VjqY-Rdcj-Kemn-I88r-V9hgJj
  manoeuvrevg2      15   1   0 wz--n-   <2.93t       0  Ghrsc1-a3Rp-AvOx-flNz-WxUS-LbTt-BXfu5Z
  oraclevg           1   6   0 wz--n- <200.00g  <53.00g 2vZ1mF-d6BA-aE7S-R929-B00x-mdh1-wb05Tx
  rootvg             1   8   0 wz--n- <205.39g  187.91g D92t9d-1VOd-fBYy-ErdZ-cvp1-1J0G-VrugJY
[root@gerdcwora1pdb01 ~]$
[root@gerdcwora1pdb01 ~]$
[root@gerdcwora1pdb01 ~]$ vgremove g7ovuA-7dLz-xBH9-UMvX-rKkm-fXVv-rpC5Ch manoeuvrevg1
  Volume group "g7ovuA-7dLz-xBH9-UMvX-rKkm-fXVv-rpC5Ch" not found
  Cannot process volume group g7ovuA-7dLz-xBH9-UMvX-rKkm-fXVv-rpC5Ch
  Couldn't find device with uuid HwdVE1-qX25-afEh-826D-jsTz-BhYP-czmphj.
  Couldn't find device with uuid YgZMMp-E3bj-seOn-sZ51-yj8G-uI2p-7n8oCf.
  Couldn't find device with uuid IdCvLh-qhcS-3DVM-WAfd-pKEs-H13P-0wGy3Q.
  Couldn't find device with uuid XdLxSW-Lqys-gOTm-etDG-gn0Q-ZikX-YYiZee.
  Couldn't find device with uuid 3YzWwB-OrUC-VObv-OfVm-oqQ5-kp5h-q3Jrhb.
  Couldn't find device with uuid qX3OWD-SbON-uhNf-IJkw-PFOU-02Z0-WU9LNc.
  Couldn't find device with uuid 4l7vq9-ofaQ-omTS-au8v-EHdG-dqS1-YGSCpy.
  Couldn't find device with uuid P7kxSI-Srgj-ATMM-DkDL-a4Xh-S2Al-YEKlZu.
  Couldn't find device with uuid qc68ka-bTwE-ia11-SIAL-5pfL-hi10-Fe0CfV.
  Couldn't find device with uuid ITcdBU-EGyz-cx0W-OBCN-6Pbe-4dg2-84s3qo.
  Couldn't find device with uuid PhgD0G-ay2R-7e0h-4ZBj-z45Q-ptd8-ywUO0i.
  Couldn't find device with uuid WDfaFC-ldPa-pbub-QdCY-m8kb-yIK9-j8Byzu.
  Couldn't find device with uuid Td7uJO-nUld-Xm4r-oTzO-15zG-McRk-Ti5D0X.
  Couldn't find device with uuid q78InL-XaEU-yLC1-ef6r-cU37-ZyVp-y13Ibj.
  Couldn't find device with uuid 8sVcjf-fl0V-kHZX-MAOl-AIt1-ggl8-z0Ywns.
  Volume group "manoeuvrevg1" not found, is inconsistent or has PVs missing.
  Consider vgreduce --removemissing if metadata is inconsistent.
[root@gerdcwora1pdb01 ~]$

[root@gerdcwora1pdb01 ~]$ vgremove manoeuvrevg1
  Couldn't find device with uuid HwdVE1-qX25-afEh-826D-jsTz-BhYP-czmphj.
  Couldn't find device with uuid YgZMMp-E3bj-seOn-sZ51-yj8G-uI2p-7n8oCf.
  Couldn't find device with uuid IdCvLh-qhcS-3DVM-WAfd-pKEs-H13P-0wGy3Q.
  Couldn't find device with uuid XdLxSW-Lqys-gOTm-etDG-gn0Q-ZikX-YYiZee.
  Couldn't find device with uuid 3YzWwB-OrUC-VObv-OfVm-oqQ5-kp5h-q3Jrhb.
  Couldn't find device with uuid qX3OWD-SbON-uhNf-IJkw-PFOU-02Z0-WU9LNc.
  Couldn't find device with uuid 4l7vq9-ofaQ-omTS-au8v-EHdG-dqS1-YGSCpy.
  Couldn't find device with uuid P7kxSI-Srgj-ATMM-DkDL-a4Xh-S2Al-YEKlZu.
  Couldn't find device with uuid qc68ka-bTwE-ia11-SIAL-5pfL-hi10-Fe0CfV.
  Couldn't find device with uuid ITcdBU-EGyz-cx0W-OBCN-6Pbe-4dg2-84s3qo.
  Couldn't find device with uuid PhgD0G-ay2R-7e0h-4ZBj-z45Q-ptd8-ywUO0i.
  Couldn't find device with uuid WDfaFC-ldPa-pbub-QdCY-m8kb-yIK9-j8Byzu.
  Couldn't find device with uuid Td7uJO-nUld-Xm4r-oTzO-15zG-McRk-Ti5D0X.
  Couldn't find device with uuid q78InL-XaEU-yLC1-ef6r-cU37-ZyVp-y13Ibj.
  Couldn't find device with uuid 8sVcjf-fl0V-kHZX-MAOl-AIt1-ggl8-z0Ywns.
  Volume group "manoeuvrevg1" not found, is inconsistent or has PVs missing.
  Consider vgreduce --removemissing if metadata is inconsistent.
[root@gerdcwora1pdb01 ~]$


We asked them to disable lvmetad, and try pvscan,vgscan, but they continued seeing this problem on 4 nodes out of 5.

1 "good" node in this 5 node cluster doesn't have this issue, it sees,


=> On gerdcwora1pdb03

  VG           Attr   Ext   #PV #LV #SN VSize    VFree    VG UUID                                VProfile #VMda VMdaFree  VMdaSize  #VMdaUse VG Tags
  applivg      wz--n- 4.00m   3   0   0 <599.99g <599.99g 7l3Dp6-w60R-ZP1t-tGzc-0jSX-kuls-5VXATh              3        0   1020.00k        3        
  manoeuvrevg1 wz--n- 4.00m  15   1   0   <2.93t       0  XOrFC0-rdEC-VjqY-Rdcj-Kemn-I88r-V9hgJj             15        0   1020.00k       15        
  manoeuvrevg2 wz--n- 4.00m  15   1   0   <2.93t       0  Ghrsc1-a3Rp-AvOx-flNz-WxUS-LbTt-BXfu5Z             15        0   1020.00k       15        
  oraclevg     wz--n- 4.00m   1   6   0 <200.00g  <87.00g IB3hGl-VPpB-RZhL-dH2O-lqKN-ah6l-SApuH0              1        0   1020.00k        1        
  rootvg       wz--n- 4.00m   1   8   0 <205.39g <188.89g xiVCaF-AhkL-0W1q-LNih-UvvL-GQ2s-6i4iq8              1        0   1020.00k        1   

Here the problematic VG is not seen.

On the bad nodes, with ALL MISSING pv's, somehow the VG seemed partially activated, and vgscan seems to indicate its picking up the metadata from the PV meant for the VG we renamed to manoeuvrevg1_bpk.




[root@nkshirsa sosreport-gerdcwora1jdb01-2019-07-16-mgqibpm]# cat sos_commands/lvm2/vgs_-v_-o_vg_mda_count_vg_mda_free_vg_mda_size_vg_mda_used_count_vg_tags_--config_global_locking_type_0 
    Reloading config files
  WARNING: Locking disabled. Be careful! This could corrupt your metadata.
  Couldn't find device with uuid HwdVE1-qX25-afEh-826D-jsTz-BhYP-czmphj.
  Couldn't find device with uuid YgZMMp-E3bj-seOn-sZ51-yj8G-uI2p-7n8oCf.
  Couldn't find device with uuid IdCvLh-qhcS-3DVM-WAfd-pKEs-H13P-0wGy3Q.
  Couldn't find device with uuid XdLxSW-Lqys-gOTm-etDG-gn0Q-ZikX-YYiZee.
  Couldn't find device with uuid 3YzWwB-OrUC-VObv-OfVm-oqQ5-kp5h-q3Jrhb.
  Couldn't find device with uuid qX3OWD-SbON-uhNf-IJkw-PFOU-02Z0-WU9LNc.
  Couldn't find device with uuid 4l7vq9-ofaQ-omTS-au8v-EHdG-dqS1-YGSCpy.
  Couldn't find device with uuid P7kxSI-Srgj-ATMM-DkDL-a4Xh-S2Al-YEKlZu.
  Couldn't find device with uuid qc68ka-bTwE-ia11-SIAL-5pfL-hi10-Fe0CfV.
  Couldn't find device with uuid ITcdBU-EGyz-cx0W-OBCN-6Pbe-4dg2-84s3qo.
  Couldn't find device with uuid PhgD0G-ay2R-7e0h-4ZBj-z45Q-ptd8-ywUO0i.
  Couldn't find device with uuid WDfaFC-ldPa-pbub-QdCY-m8kb-yIK9-j8Byzu.
  Couldn't find device with uuid Td7uJO-nUld-Xm4r-oTzO-15zG-McRk-Ti5D0X.
  Couldn't find device with uuid q78InL-XaEU-yLC1-ef6r-cU37-ZyVp-y13Ibj.
  Couldn't find device with uuid 8sVcjf-fl0V-kHZX-MAOl-AIt1-ggl8-z0Ywns.
    Couldn't find device with uuid HwdVE1-qX25-afEh-826D-jsTz-BhYP-czmphj.
    Couldn't find device with uuid YgZMMp-E3bj-seOn-sZ51-yj8G-uI2p-7n8oCf.
    Couldn't find device with uuid IdCvLh-qhcS-3DVM-WAfd-pKEs-H13P-0wGy3Q.
    Couldn't find device with uuid XdLxSW-Lqys-gOTm-etDG-gn0Q-ZikX-YYiZee.
    Couldn't find device with uuid 3YzWwB-OrUC-VObv-OfVm-oqQ5-kp5h-q3Jrhb.
    Couldn't find device with uuid qX3OWD-SbON-uhNf-IJkw-PFOU-02Z0-WU9LNc.
    Couldn't find device with uuid 4l7vq9-ofaQ-omTS-au8v-EHdG-dqS1-YGSCpy.
    Couldn't find device with uuid P7kxSI-Srgj-ATMM-DkDL-a4Xh-S2Al-YEKlZu.
    Couldn't find device with uuid qc68ka-bTwE-ia11-SIAL-5pfL-hi10-Fe0CfV.
    Couldn't find device with uuid ITcdBU-EGyz-cx0W-OBCN-6Pbe-4dg2-84s3qo.
    Couldn't find device with uuid PhgD0G-ay2R-7e0h-4ZBj-z45Q-ptd8-ywUO0i.
    Couldn't find device with uuid WDfaFC-ldPa-pbub-QdCY-m8kb-yIK9-j8Byzu.
    Couldn't find device with uuid Td7uJO-nUld-Xm4r-oTzO-15zG-McRk-Ti5D0X.
    Couldn't find device with uuid q78InL-XaEU-yLC1-ef6r-cU37-ZyVp-y13Ibj.
    Couldn't find device with uuid 8sVcjf-fl0V-kHZX-MAOl-AIt1-ggl8-z0Ywns.
    There are 15 physical volumes missing.
  VG               Attr   Ext   #PV #LV #SN VSize    VFree    VG UUID                                VProfile #VMda VMdaFree  VMdaSize  #VMdaUse VG Tags
  applivg          wz--n- 4.00m   3   3   0 <599.99g <446.99g prTh5H-qz53-22S9-WINL-zJ7U-Qc5J-ti5FRD              3   507.50k  1020.00k        3        
  manoeuvrevg1     wz-pn- 4.00m  15   0   0   <2.93t   <2.93t g7ovuA-7dLz-xBH9-UMvX-rKkm-fXVv-rpC5Ch              1   506.00k  1020.00k        1        
  manoeuvrevg1_bkp wz--n- 4.00m  15   1   0   <2.93t       0  XOrFC0-rdEC-VjqY-Rdcj-Kemn-I88r-V9hgJj             14   504.00k  1020.00k       14        
  manoeuvrevg2     wz--n- 4.00m  15   1   0   <2.93t       0  Ghrsc1-a3Rp-AvOx-flNz-WxUS-LbTt-BXfu5Z             15   504.00k  1020.00k       15        
  oraclevg         wz--n- 4.00m   1   7   0 <300.00g <154.00g QTYo03-fGma-O6yI-REy3-sk2C-f89M-gisLQJ              1   506.00k  1020.00k        1        
  rootvg           wz--n- 4.00m   1   8   0 <205.39g <188.84g dMoyeX-Jd62-spdH-owqa-O1SU-BDUT-5zyX6j              1   506.00k  1020.00k        1        
    Reloading config files
[root@nkshirsa sosreport-gerdcwora1jdb01-2019-07-16-mgqibpm]# 


What seems to have happened is, after the rename, the manoeuvrevg1 VG has picked up metadata from a PV belonging to the manoeuvrevg1_bkp VG. (/dev/mapper/manoeuvrevg1d1 )

Supporting data:

in sosreport-gerdcwora1jdb01-2019-07-16-mgqibpm


vgscan shows,

      Processing VG manoeuvrevg1 g7ovuA-7dLz-xBH9-UMvX-rKkm-fXVv-rpC5Ch <== 
        Dropping cache for manoeuvrevg1.
      Reading VG manoeuvrevg1 g7ovuA-7dLz-xBH9-UMvX-rKkm-fXVv-rpC5Ch
        Rescanning devices for manoeuvrevg1
        lvmcache has no info for vgname "manoeuvrevg1" with VGID g7ovuA7dLzxBH9UMvXrKkmfXVvrpC5Ch.
        Scanning 1 devices for VG info
        Scanning submitted 1 reads
        Processing data from device /dev/mapper/manoeuvrevg1d1 253:446 fd 4 block 0x559d34176b50 <==
      /dev/mapper/manoeuvrevg1d1: lvm2 label detected at sector 1
        lvmcache /dev/mapper/manoeuvrevg1d1: now in VG #orphans_lvm2 (#orphans_lvm2) with 0 mda(s).
        /dev/mapper/manoeuvrevg1d1: PV header extension version 2 found
        Reading mda header sector from /dev/mapper/manoeuvrevg1d1 at 4096
        Reading metadata summary from /dev/mapper/manoeuvrevg1d1 at 28672 size 3205 (+0)
        Found metadata summary on /dev/mapper/manoeuvrevg1d1 at 28672 size 3205 for VG manoeuvrevg1
        lvmcache has no info for vgname "manoeuvrevg1" with VGID g7ovuA7dLzxBH9UMvXrKkmfXVvrpC5Ch.
        lvmcache has no info for vgname "manoeuvrevg1".
        lvmcache /dev/mapper/manoeuvrevg1d1: now in VG manoeuvrevg1 with 1 mda(s).
        lvmcache /dev/mapper/manoeuvrevg1d1: VG manoeuvrevg1: set VGID to g7ovuA7dLzxBH9UMvXrKkmfXVvrpC5Ch.
        lvmcache /dev/mapper/manoeuvrevg1d1: VG manoeuvrevg1: set seqno to 3
        lvmcache /dev/mapper/manoeuvrevg1d1: VG manoeuvrevg1: set mda_checksum to 316583be mda_size to 3205
        lvmcache /dev/mapper/manoeuvrevg1d1: VG manoeuvrevg1: set creation host to gerdcwora1jdb02.
        Scanned devices: read errors 0 process errors 0 failed 0
        Reading VG manoeuvrevg1 from /dev/mapper/manoeuvrevg1d1      <----- this PV is of manoeuvrevg1_bkp
        Reading mda header sector from /dev/mapper/manoeuvrevg1d1 at 4096
        Reading metadata from /dev/mapper/manoeuvrevg1d1 at 28672 size 3205 (+0)
        Allocated VG manoeuvrevg1 at 0x559d3843aa90.
        No device with uuid HwdVE1qX25afEh826DjsTzBhYPczmphj.
  Couldn't find device with uuid HwdVE1-qX25-afEh-826D-jsTz-BhYP-czmphj.
        No device with uuid YgZMMpE3bjseOnsZ51yj8GuI2p7n8oCf.
  Couldn't find device with uuid YgZMMp-E3bj-seOn-sZ51-yj8G-uI2p-7n8oCf.
        No device with uuid IdCvLhqhcS3DVMWAfdpKEsH13P0wGy3Q.
  Couldn't find device with uuid IdCvLh-qhcS-3DVM-WAfd-pKEs-H13P-0wGy3Q.
        No device with uuid XdLxSWLqysgOTmetDGgn0QZikXYYiZee.
  Couldn't find device with uuid XdLxSW-Lqys-gOTm-etDG-gn0Q-ZikX-YYiZee.

note also:

        Found metadata on /dev/mapper/manoeuvrevg1d1 at 28672 size 3205 for VG manoeuvrevg1
        Cached VG manoeuvrevg1 had incorrect PV list <----
        Freeing VG manoeuvrevg1 at 0x559d3843aa90.
        Reading mda header sector from /dev/mapper/manoeuvrevg1d1 at 4096
        Reading metadata from /dev/mapper/manoeuvrevg1d1 at 28672 size 3205 (+0)
        Allocated VG manoeuvrevg1 at 0x559d3843aa90.
        No device with uuid HwdVE1qX25afEh826DjsTzBhYPczmphj.


The PV is this multipath device.

manoeuvrevg1d1 (360000970000297900076533030303244) dm-446 EMC     ,SYMMETRIX
size=200G features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  |- 2:0:0:523  sdakk              132:960  active ready running
  |- 2:0:1:523  sdamy              8:1248   active ready running
  |- 16:0:0:523 sdaib              128:1008 active ready running
  `- 16:0:1:523 sdafi              68:896   active ready running


As we see, its a PV for manoeuvrevg1_bkp.

[root@nkshirsa sosreport-gerdcwora1jdb01-2019-07-16-mgqibpm]# cat sos_commands/lvm2/pvs_-a_-v_-o_pv_mda_free_pv_mda_size_pv_mda_count_pv_mda_used_count_pe_start_--config_global_locking_type_0 | grep manoeuvrevg1d1

  /dev/mapper/manoeuvrevg1d1             manoeuvrevg1_bkp lvm2 a--  <200.00g       0   200.00g qUB1xp-Oi49-A000-yt6d-QJc8-rsB0-a4hH2R   506.00k  1020.00k     1        1   1.00m

We are wondering how this has happened, could it be a bug in the rename?

Would it be dangerous now to try vgremove --force of manoeuvrevg1, would it end up removing manoeuvrevg1_bkp since it seems to be pointing to the PV meant for manoeuvrevg1_bpk ! 

I will add more details in the additional details.

Version-Release number of selected component (if applicable):

lvm2-2.02.180-8.el7.x86_64                                  Mon Mar  4 14:10:27 2019
lvm2-libs-2.02.180-8.el7.x86_64                             Mon Mar  4 14:10:26 2019



Additional info:

Attaching vgscan -vvv to the bz.

I will also attach lvm metadata in /etc/lvm

Comment 3 nikhil kshirsagar 2019-07-17 03:35:13 UTC
Created attachment 1591241 [details]
metadata from 1 of the 5 nodes

Comment 5 nikhil kshirsagar 2019-07-17 04:00:40 UTC
Created attachment 1591254 [details]
metadata from node jdb02

Comment 6 nikhil kshirsagar 2019-07-17 04:01:59 UTC
Created attachment 1591265 [details]
verbose outputs of some lvm commands

Comment 7 David Teigland 2019-07-17 16:33:48 UTC
> At this point use_lvmetad was set to 1 in lvm.conf , they do not use clvmd
>
> for this 5 node RAC cluster but told us that
> 
> "
> 
> These are manœuvre LUNs (shared LUNs), it’s supposed to be created /
> declared on ALL servers but mounted on only 1 server at once. 
> This must not be mounted on multiple servers in the same time, for sure. 
> This will be used to copy DB export from 1 server (A) to another (B) using
> below process : 
> FS /manoeuvre1 and /manoeuvre2 are mounted on server A 
> DB export is copied to FS /manoeuvre1 and /manoeuvre2 on server A 
> FS /manoeuvre1 and /manoeuvre2 are unmounted on server A 
> FS /manoeuvre1 and /manoeuvre2 are mounted on server B 
> DB export is copied from FS /manoeuvre1 and /manoeuvre2 on server B 
> FS /manoeuvre1 and /manoeuvre2 are unmounted on server B

This is highly incorrect and improper use of lvm.  The customer should:

- Disable lvmetad immediately and permanently.  lvmetad is incompatible with shared storage; the problems they have seen already are  only the beginning.

- Enable system ID on every host that can see the shared storage, and assign system IDs to each VG on the shared storage.  See lvmsystemid(7) for more information about using system ID.

- To move the VG between servers, run vgexport on the source server, then run vgimport on the destination server.

- If they want different servers to use LVs in a single VG simultaneously, the only supported way to do this is with lvmlockd or clvmd.

Comment 8 David Teigland 2019-07-17 16:44:36 UTC
> # vgrename uuid manoeuvrevg1_bkp
> 
> "
> 
> On hindsight it may have been better to do a vgremove here instead of a
> rename?

Once you have disabled lvmetad and are using system IDs, then duplicate VG names are only a problem if both VGs are visible on a single host.  If that's the case, then the only thing you can do to resolve it is vgrename <uuid> <newname>, vgremove will not be allowed (it will not know which VG you mean to remove.)

Comment 9 David Teigland 2019-07-17 16:58:20 UTC
> So what they requested was, to know how to remove the duplicated manoeuvrevg1
> 
> we want to remove that extra VG(which is duplicated ) in this server.
> 
> VG           Attr   Ext   #PV #LV #SN VSize    VFree    VG UUID
> manoeuvrevg1 wz-pn- 4.00m  15   1   0   <2.93t       0 
> g7ovuA-7dLz-xBH9-UMvX-rKkm-fXVv-rpC5Ch
> manoeuvrevg1 wz-pn- 4.00m  15   1   0   <2.93t       0 
> XOrFC0-rdEC-VjqY-Rdcj-Kemn-I88r-V9hgJj
> manoeuvrevg2 wz-pn- 4.00m  15   1   0   <2.93t       0 
> FtAV6z-TlT1-fpBU-bfsQ-3t0f-Eq8V-v2z64W

What do you mean by "want to remove"?  Do you want it to not be visible, or do you want to wipe the disks, or do you want it to be visible with a different name?

Whichever case it is, to start repairing things you need to first detach the storage from all but one host, and disable lvmetad everywhere.  Then you need to figure out which devices (/dev/sdX) belong to each of the different VGs.  Then you need to set up the filter in lvm.conf to exclude all of the devices from the VG you are not going to work on.  Then try to repair and rename the VG on the remaining devices (or wipe them if you don't want that VG to exist any more.)  Then change the filter to exclude the devices you've just repaired or wiped, and include the devices you excluded before, and repeat the process of repairing or wiping the VG on those devices.


Note You need to log in before you can comment on or make changes to this bug.