First of all, one should not think of this problem in terms of VMs. I am actually more interested in bare metal hardware (where one is perhaps also more likely to encounter changing disk IDs). This also makes any virt-sysprep-like solutions unfeasible.

My use case is a disaster recovery tool called ReaR, which makes a backup of a system (including a description of all the storage layout). When one needs to recover a system, one boots a special rescue medium and ReaR recreates the storage layout, mounts the filesystems and restores the backup data to them. Since one will often need to do it when a hard disk has failed and is replaced, this involves changing disk IDs and is similar to disk cloning.

After the recovered system boots, it can not see the logical volumes (bz2145014), because /etc/lvm/devices/system.devices is from the data backup (the old system) and has the old WWIDs. A simple workaround is to remove /etc/lvm/devices/system.devices. This can be automated inside ReaR but I think it is not ideal because it will be some degradation of functionality in the cloned system relative to the original system (If not, I would argue that the devices file is not very useful.)

One could somehow add the entries for the new disks to the devices file (vgimportdevices is one way to do it), but the devices file will now contain entries both for the new and old disk for the same PVID and the LVM commands will print warnings as they encounter this situation, which is not ideal. Hence the request for vgimportdevices to remove stale PVID entries (stale = those that do not have a disk but there is another entry with the same PVID that has a disk).

There are probably other ways to do it, like add this functionality to lvmdevices --update (this would match its description, "Update the devices file to fix incorrect values").

There are other ways to get to the same situation, basically it will happen every time one clones the OS installation to new disks (bz2135427).

Thanks, that makes it much clearer.  You've also mentioned that the original system.devices may contain devices that are not attached at the time that local VGs are recreated, but those entries should remain in the file.

Here's another possibility.  It sounds like you know the PVIDs from the original system, and are then using pvcreate --uuid to recreate PVs using the same PVID that existed before.  In this case, you could run lvmdevices --delpvid <pvid> followed by pvcreate --uuid <pvid>.  The lvmdevices command would first remove the system.devices entry for the given PVID and then pvcreate will recreate the PV on disk with the same PVID.  The pvcreate command automatically adds an entry for the new PV to system.devices.  It will have that same PVID but use the WWID (or other device id type) for the new hw device.

(In reply to David Teigland from comment #2)
> Thanks, that makes it much clearer.  You've also mentioned that the original
> system.devices may contain devices that are not attached at the time that
> local VGs are recreated, but those entries should remain in the file.

Yes, during the system recover it is likely that disks that don't need to be recovered will be disconnected (to reduce the risk that the recovery procedure will accidentally reformat them).

> Here's another possibility.  It sounds like you know the PVIDs from the
> original system, and are then using pvcreate --uuid to recreate PVs using
> the same PVID that existed before.  In this case, you could run lvmdevices
> --delpvid <pvid> followed by pvcreate --uuid <pvid>.  The lvmdevices command
> would first remove the system.devices entry for the given PVID and then
> pvcreate will recreate the PV on disk with the same PVID.  The pvcreate
> command automatically adds an entry for the new PV to system.devices.  It
> will have that same PVID but use the WWID (or other device id type) for the
> new hw device.

I can not do that because when pvcreate --uuid runs, the original devices file is not yet there - it gets extracted from the backup only after all the storage gets recreated and moutned (in a chroot).

What happens is that pvcreate --uuid runs in a ramdisk (the rescue environment). So it creates /etc/lvm/devices/system.devices in the ramdisk and it has correct entries. I could then somehow merge this correct but incomplete devices file with the original /etc/lvm/devices/system.devices extracted from the backup ("merge" means that entries in the original /etc/lvm/devices/system.devices that also exist in the new /etc/lvm/devices/system.devices would get replaced by the new ones).

But I don't know how to do that easily programmatically - I would need to get a list of PVIDs in the new device file and remove them from the old device file, then append the new device file to the old. lvmdevices in the rescue system prints the device entries from the devices file, but the output is not easily machine readable:
  Device /dev/sdc3 IDTYPE=sys_wwid IDNAME=naa.5000c500b7ba24f3 DEVNAME=/dev/sdc3 PVID=olf2flGBlOcToi5p3o0W0LsToUIDE2GC PART=3
it even goes to stderr, not stdout, indicating that it is intended for the user and not for processing.

OK, it sounds like you're working with the list of PVIDs (used for pvcreate), the old system.devices (from backup), and the new system.devices (from ramdisk.)  Could you just loop through the PVIDs, and for each PVID, replace the entry in old system.devices with the corresponding entry from new system.devices?


(In reply to Pavel Cahyna from comment #3)
> What happens is that pvcreate --uuid runs in a ramdisk (the rescue
> environment). So it creates /etc/lvm/devices/system.devices in the ramdisk
> and it has correct entries. I could then somehow merge this correct but
> incomplete devices file with the original /etc/lvm/devices/system.devices
> extracted from the backup ("merge" means that entries in the original
> /etc/lvm/devices/system.devices that also exist in the new
> /etc/lvm/devices/system.devices would get replaced by the new ones).

yes, that sounds good.

> But I don't know how to do that easily programmatically - I would need to
> get a list of PVIDs in the new device file and remove them from the old
> device file, then append the new device file to the old.

For each PVID in the new system.devices, replace the corresponding line in old system.devices with the line in new system.devices?

(In reply to David Teigland from comment #4)
> OK, it sounds like you're working with the list of PVIDs (used for
> pvcreate), the old system.devices (from backup), and the new system.devices
> (from ramdisk.)  Could you just loop through the PVIDs, and for each PVID,
> replace the entry in old system.devices with the corresponding entry from
> new system.devices?
> 
> 
> (In reply to Pavel Cahyna from comment #3)
> > What happens is that pvcreate --uuid runs in a ramdisk (the rescue
> > environment). So it creates /etc/lvm/devices/system.devices in the ramdisk
> > and it has correct entries. I could then somehow merge this correct but
> > incomplete devices file with the original /etc/lvm/devices/system.devices
> > extracted from the backup ("merge" means that entries in the original
> > /etc/lvm/devices/system.devices that also exist in the new
> > /etc/lvm/devices/system.devices would get replaced by the new ones).
> 
> yes, that sounds good.
> 
> > But I don't know how to do that easily programmatically - I would need to
> > get a list of PVIDs in the new device file and remove them from the old
> > device file, then append the new device file to the old.
> 
> For each PVID in the new system.devices, replace the corresponding line in
> old system.devices with the line in new system.devices?

Yes. This requires me to write a parser for system.devices, which will break if the format changes or there is something unexpected in the file (is the grammar documented anywhere?)

I wouldn't think you'd need a full parser, just grep the file for PVID=<uuid> and use the full resulting line.  If the format completely changes then there would be ways to deal with it.

But I don't need to find a device with a specific PVID in the file - I need to enumerate all the PVIDs in the new devices file in order to be able to remove them from the old devices file using lvmdevices --delpvid. I don't see how to easily enumerate them. I could use pvdisplay to list all PVIDs and check which exist in the devices file, but:

# pvs -o +pv_uuid
  PV         VG                    Fmt  Attr PSize   PFree PV UUID                               
  /dev/sdb3  rhel_hpe-dl120gen9-01 lvm2 a--  929.92g    0  KtAgll-kufp-0CJz-EVLz-qliW-BKkt-dss0ZU
# cat /etc/lvm/devices/system.devices
# LVM uses devices listed in this file.
# Created by LVM command vgs pid 1347 at Thu May 18 10:22:48 2023
VERSION=1.1.5
IDTYPE=sys_wwid IDNAME=naa.5000c500798fc371 DEVNAME=/dev/sdb3 PVID=KtAgllkufp0CJzEVLzqliWBKktdss0ZU PART=3


- there are two different formats, which is not favorable for using text-based tools on the file.

> What happens is that pvcreate --uuid runs in a ramdisk (the rescue environment). So it creates /etc/lvm/devices/system.devices in the ramdisk and it has correct entries.

Here it sounds like you know the list of PVIDs, because you are using them with pvcreate --uuid.  The PVIDs you use with pvcreate will exist in the new system.devices as PVID=<uuid>.

> I need to enumerate all the PVIDs in the new devices file

Here it sounds like you don't know the list of PVIDs.  So I'm confused about if/when you know the PVIDs.  If you don't have access to the list of PVIDs you used with pvcreate above, then you could get them from the new system.devices file, which would require just enough parsing to get the PVID=<uuid> from each line.

I know the PVIDs in this case, but I should still verify that they exist in the devices file - removing the entry from the old devices file when it does not exist in the new devices file either would leave LVM broken.

Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.