Bug 1660625

Summary:	The pvcreate command fails with "Metadata location has invalid summary for VG" error
Product:	Red Hat Enterprise Linux 7	Reporter:	jhouston
Component:	lvm2	Assignee:	David Teigland <teigland>
lvm2 sub component:	LVM Metadata / lvmetad	QA Contact:	cluster-qe <cluster-qe>
Status:	CLOSED WONTFIX	Docs Contact:
Severity:	high
Priority:	high	CC:	aborniak, agk, bgamari, cmarthal, frosae, heinzm, jbrassow, jpittman, loberman, msnitzer, mtowey, prajnoha, rahul00akolkar, revers, stanislav.polasek, teigland, zkabelac
Version:	7.6	Keywords:	Triaged
Target Milestone:	rc
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-08-28 15:06:47 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1711360

Description jhouston 2018-12-18 20:39:36 UTC

Description of problem:

Unable to see PV after reboot.

While attempting a restore of metadata seeing failure:

# pvcreate --uuid="<UUID>" --restorefile  /etc/lvm/backup/vg /dev/sdx
  Couldn't find device with uuid <UUID>.
  /dev/sdx: Checksum error at offset 7168
  Couldn't read volume group metadata from /dev/sdx.
  Metadata location on /dev/sdx at 7168 has invalid summary for VG.
  Failed to read metadata summary from /dev/sdx
  Failed to scan VG from /dev/sdx

Version-Release number of selected component (if applicable):

lvm2-2.02.180-10.el7_6.2.x86_64                             Fri Dec 14 21:02:00 2018
lvm2-libs-2.02.180-10.el7_6.2.x86_64                        Fri Dec 14 21:00:39 2018

How reproducible:

Issue is intermittently seen in different environments

Steps to Reproduce:
Unknown currently

Actual results:

Seeing checksum error offsets reported by LVM

Expected results:

Expect PV to be able to be read, and restore/fix features to ignore checksum errors

Comment 16 Jonathan Earl Brassow 2019-08-28 15:06:47 UTC

We have a solution for this problem upstream and in RHEL8.  Unfortunately, it is too invasive to consider back porting to RHEL7 at this stage.  We will have to address cases as they come individually.  If this is a widespread and frequent problem (which it does not appear to be), perhaps there could be a workaround found.

Comment 17 Frode Sætre 2019-10-08 07:46:59 UTC

Any solution ?

Comment 18 aborniak 2019-10-09 07:21:28 UTC

I faced with same issue. 
In our case it has been fixed in following way: 
Boot from CentOS 6 and run "pvck" for affected physical volume. 
Then just run "vgcfgrestore".

Comment 19 Frode Sætre 2019-10-09 07:34:57 UTC

Thank you, we solved it the same way.

Comment 22 aborniak 2021-12-24 17:17:17 UTC

I was just faced with this issue again. 
Read my comment (2019-10-09 07:21:28 UTC) and fix it the same way as before.

Comment 23 Rahul 2021-12-29 13:21:23 UTC

I'm facing a same issue. Error as follows-
# pvs
  /dev/mapper/mpathd: Checksum error at offset 35328
  Couldn't read volume group metadata from /dev/mapper/mpathd.
  Metadata location on /dev/mapper/mpathd at 35328 has invalid summary for VG.
  Failed to read metadata summary from /dev/mapper/mpathd
  Failed to scan VG from /dev/mapper/mpathd
  /dev/mapper/mpathb: Checksum error at offset 35328
  Couldn't read volume group metadata from /dev/mapper/mpathb.
  Metadata location on /dev/mapper/mpathb at 35328 has invalid summary for VG.
  Failed to read metadata summary from /dev/mapper/mpathb
  Failed to scan VG from /dev/mapper/mpathb
  /dev/mapper/mpathc: Checksum error at offset 35328
  Couldn't read volume group metadata from /dev/mapper/mpathc.
  Metadata location on /dev/mapper/mpathc at 35328 has invalid summary for VG.
  Failed to read metadata summary from /dev/mapper/mpathc
  Failed to scan VG from /dev/mapper/mpathc

I removed the LV/VG/PV. While trying with "vgcfgrestore" getting error /etc/lvm/backup/nfsvg : no such file or directory.

Comment 24 Ben Gamari 2024-04-26 17:55:37 UTC

Unfortunately I am seeing a very similar issue with:

$ lvchange --version
  LVM version:     2.03.21(2) (2023-04-21)
  Library version: 1.02.195 (2023-04-21)
  Driver version:  4.45.0
  Configuration:   <removed>
$ pvcreate \
        -ff \
        --uuid VnN10r-D0T4-HOmi-P7wI-6Fce-XvX6-2BZHd2 \
        --restorefile /etc/lvm/archive/data_00002-925283222.vg \
        /dev/loop0p1
  WARNING: Couldn't find device with uuid nC8H8D-X3WR-h5Ia-h4Oh-i1xA-ech4-B2sNzq.
  WARNING: Couldn't find device with uuid McragJ-UsLP-m4ap-RyC3-fV9t-lksv-ztcGdz.
  WARNING: Couldn't find device with uuid 31Akqw-s5gq-X4Uv-kJ56-Kw7L-m8rS-WLIFpH.
  WARNING: Couldn't find device with uuid VnN10r-D0T4-HOmi-P7wI-6Fce-XvX6-2BZHd2.
  WARNING: wrong checksum 0 in mda header on /dev/loop0p1 at 4096
  WARNING: wrong magic number in mda header on /dev/loop0p1 at 4096
  WARNING: wrong version 0 in mda header on /dev/loop0p1 at 4096
  WARNING: wrong start sector 0 in mda header on /dev/loop0p1 at 4096
  WARNING: bad metadata header on /dev/loop0p1 at 4096.
  WARNING: scanning /dev/loop0p1 mda1 failed to read metadata summary.
  WARNING: repair VG metadata on /dev/loop0p1 with vgck --updatemetadata.
  WARNING: scan failed to get metadata summary from /dev/loop0p1 PVID VnN10rD0T4HOmiP7wI6FceXvX62BZHd2
  WARNING: wrong checksum 0 in mda header on /dev/loop0p1 at 4096
  WARNING: wrong magic number in mda header on /dev/loop0p1 at 4096
  WARNING: wrong version 0 in mda header on /dev/loop0p1 at 4096
  WARNING: wrong start sector 0 in mda header on /dev/loop0p1 at 4096
  WARNING: bad metadata header on /dev/loop0p1 at 4096.
  WARNING: scanning /dev/loop0p1 mda1 failed to read metadata summary.
  WARNING: repair VG metadata on /dev/loop0p1 with vgck --updatemetadata.
  WARNING: scan failed to get metadata summary from /dev/loop0p1 PVID VnN10rD0T4HOmiP7wI6FceXvX62BZHd2
Really INITIALIZE physical volume "/dev/loop0p1" of volume group "<unknown>" [y/n]? y
  WARNING: Forcing physical volume creation on /dev/loop0p1 of volume group "<unknown>"
  WARNING: wrong checksum 0 in mda header on /dev/loop0p1 at 4096
  WARNING: wrong magic number in mda header on /dev/loop0p1 at 4096
  WARNING: wrong version 0 in mda header on /dev/loop0p1 at 4096
  WARNING: wrong start sector 0 in mda header on /dev/loop0p1 at 4096
  WARNING: bad metadata header on /dev/loop0p1 at 4096.
  WARNING: scanning /dev/loop0p1 mda1 failed to read metadata summary.
  WARNING: repair VG metadata on /dev/loop0p1 with vgck --updatemetadata.
  WARNING: scan failed to get metadata summary from /dev/loop0p1 PVID VnN10rD0T4HOmiP7wI6FceXvX62BZHd2
  Command failed with status code 5.

I suspect the fix referenced above is https://gitlab.com/lvmteam/lvm2/-/commit/ba7ff96faff052c6145c71222ea5047a6bcee33b and surrounding commits, so this is likely a slightly different problem, despite its similar symptoms.

Comment 25 Zdenek Kabelac 2024-04-28 13:54:51 UTC

Please collect 1st. MiB of you device.

Attach  -vvvv trace from a pvcreate command.

And just try to use  'wipefs -a  /dev/loop0p1'  and retry your command.
(eventually you may try  'pvremove -ff' first)

And likely open a new BZ  (and refer this old closed one)

Comment 26 Red Hat Bugzilla 2024-08-27 04:25:03 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days