Bug 2217510

Summary:	RFE: default to creating lvmdevices file on RPM install
Product:	Red Hat Enterprise Linux 9	Reporter:	Dusty Mabe <dustymabe>
Component:	lvm2	Assignee:	LVM Team <lvm-team>
lvm2 sub component:	Configuration files	QA Contact:	cluster-qe <cluster-qe>
Status:	NEW ---	Docs Contact:
Severity:	unspecified
Priority:	unspecified	CC:	agk, alitke, heinzm, jbrassow, msnitzer, prajnoha, teigland, travier, zkabelac
Version:	9.2	Keywords:	FutureFeature
Target Milestone:	rc
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Dusty Mabe 2023-06-26 13:32:01 UTC

Description of problem:

In RHEL 9 an LVM devices file gets created by Anaconda if the install includes an LVM storage configuration [1]. This is an RFE to make the creation of the devices file in `/etc/` always happen (at least for new installs) via RPM scriptlets instead.

[1] https://github.com/rhinstaller/anaconda/blob/6f604b55f6caa5d570764a1d236287fe7ad735ab/pyanaconda/modules/storage/installation.py#L334-L355

Background:

RHCOS (basis of OpenShift) doesn't use Anaconda to perform an installation. It does use a tool (`coreos-installer`), but the tool mostly just does a dd of a prebuilt image to the disk (basic partition+filesystems, no LVM). The pre-built image does include the LVM RPMs, though.

We are currently encountering an issue [2] where Kubevirt guests (virtual machines) where the users choose to put LVM on attached block devices is conflicting with the host (both guest and host are interacting with the LVM devices). Here are a few details:

- Running pods (containers) that start VirtualMachines.
- These VirtualMachines can have block storage attached to them.
- Users can choose to create LVM PVs out of this block storage.
- Both the Host (RHCOS Running OpenShift) and Guest see LVM signatures on these devices.
- Need a way to tell the host to completely ignore the devices attached to the guest.
- lvmdevices is relatively new, but presents an option
- RHCOS can implement a workaround/fix for this for now
- but would like to not behave differently than RHEL in the long term

For RHCOS we are planning to implement a workaround for this where upgrading nodes have a script that gets run to create an LVM devices file (either empty or with any detected LVM devices from the host), but in the long run we'd like to not differ from RHEL here, which means have a solution to this problem in the RPMs I believe.

[2] https://issues.redhat.com/browse/OCPBUGS-5223

Comment 2 David Teigland 2023-06-26 17:06:42 UTC

The basic idea of using system.devices on the host is the right idea; this is one of the specific cases that we created the devices file to address (in RHV at the time.) The hard part is the installation/initialization issues, because everything in that area is so customized these days, it's not just anaconda these days. It's likely that customized installation methods will also need custom ways to handle the devices file.

The basic logic, as you've already seen, is:
- No system.devices disables the feature, and causes the machine to fall back to using the lvm.conf filter.
- Existing but empty system.devices enables the feature, and causes the machine to see no PVs.
- pvcreate will create a new system.devices file only if there are no other PVs seen on the machine.
- lvmdevices --adddev and vgimportdevices will always create a new system.devices file.

This logic allows us to enable the devices file feature by default in RHEL9, while also allowing upgrades from RHEL8 without any disruption for users. Existing lvm.conf filters users had configured in RHEL8 will continue to work after upgrading to RHEL9. A user who wants to enable the devices file after upgrading can just run vgimportdevices -a.

For anaconda, it was simpler for them to keep the devices file feature off during installation, which meant they could just keep all their existing lvm usage unchanged. At the end they just added vgimportdevices -a to create a new system.devices from the PVs they'd created.

The suggestion to create a new, empty system.devices file from the rpm install sounds reasonable, but I worry about it breaking other cases. What if a user uninstalls and reinstalls the lvm rpm? Wouldn't that break machines that are depending on no system.devices to disable the feature? I have very little knowledge about the creative ways that rpms are used, and what expectations exist. Maybe someone with more expertise can tell us what's safe to assume in an rpm install.

I'm doubtful that we could get by with an rpm script populating system.devices. A command to do this scans disks on the system, which can be non-trivial, and I'm not sure it's safe to assume that all necessary disks are present at the time of the rpm install.

There are some other similar cases where we might like to recreate system.devices on first boot (bug 2213157, bug 2059545). Even virt-sysprep, which is meant to solve this, will just remove system.devices and doesn't have a solution for recreating system.devices. I've been thinking about some method were a temp file could be created in the image that would trigger lvm to recreate system.devices before using it. Maybe some of these ideas could be useful.

Comment 3 David Teigland 2023-07-21 18:45:04 UTC

I've created bug 2224641 describing a method for lvm to automatically generate a system.devices file for the root VG, and it may be a solution for this case.