Bug 2258764 - Commands pvs and vgscan doesn't work due to an existing file in /etc/lvm/devices [NEEDINFO]
Summary: Commands pvs and vgscan doesn't work due to an existing file in /etc/lvm/devices
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: lvm2
Version: 39
Hardware: Unspecified
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: LVM and device-mapper development team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2024-01-17 10:39 UTC by Peter Boy
Modified: 2024-02-07 20:45 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2024-01-17 10:55:44 UTC
Type: ---
Embargoed:
dominik: needinfo? (mcsontos)


Attachments (Terms of Use)

Description Peter Boy 2024-01-17 10:39:16 UTC
If you connect a new device containing a VG, the commands pvs and vgscan does not find the new VG and you are unable to add the VG to your system. As soon as you delete the file /etc/lvm/devices/system.devices, both utilities find the new VG. As soon as you restore said file.

In particular, the bug prevents the arm-image-installer utility from transferring the distribution image to a boot device if the system is working with LVM, e.g. Fedora Server. But many of the usual LVM administration tasks are also impeded.

It is a regression from 37, where the directory /etc/lvm/devices did not exist. Also not affected are systems that were upgraded via DNF ugrade plugin from F37. Although the directory is created, it is not filled with a file.  

Reproducible: Always

Steps to Reproduce:
1. Install and boot Fedora Server 
2. Add a disk containing a VG 
3. Use pvs or vgscan to find the newly added VG
Actual Results:  
Both utilities don't list the new VG

Expected Results:  
Both utilities should find the new VG and list it in the output

See bug https://bugzilla.redhat.com/show_bug.cgi?id=2246871

Comment 1 Zdenek Kabelac 2024-01-17 10:55:28 UTC
This is not a bug but a feature that was enabled.

See lvm.conf option  'use_devicesfile = 1'

It's a new/different way a solving problems how to select/filter devices lvm2 should use/see.

So when 'devicesfile' is in use - only devices enlisted in this file are scanned by lvm2.
User can use  lvmdevices command to maintain its content.

When the devicesfile file is erased or the option is disabled - lvm2 will use the 'filter=' mechanism known for ages  (that's also the reason why it started to work in your case).

It's likely debatable which behavior should be default.

Comment 2 Zdenek Kabelac 2024-01-17 10:58:11 UTC
It's worth to be mentioned the 'configure --with-default-use-devices-file=0/1' exists to allow distribution maintainer to select the right setting for their likeness.

Comment 3 Paul Howarth 2024-01-18 08:11:28 UTC
I came across this issue when reinstalling my own server with Fedora 39.
Eventually I figured out I was able to work around it by doing:

# vgimportdevices -a
# vgchange -ay

Comment 4 Dominik 'Rathann' Mierzejewski 2024-01-18 11:30:51 UTC
FYI, this was introduced in the 2.03.16 update: https://src.fedoraproject.org/rpms/lvm2/c/44a7aeaa643a5f5854414d99a3f84859e7273d88?branch=rawhide

Also, when this was introduced upstream (2.03.12), it was off by default:
https://gitlab.com/lvmteam/lvm2/-/blob/main/WHATS_NEW?ref_type=heads#L199

So, arguably, changing this default should've been at least announced
and discussed in the mailing list. A change proposal would've been
appropriate, too.

PS. Out of curiosity, why is release monitoring integration disabled for lvm2?

Comment 5 Peter Boy 2024-01-18 12:03:59 UTC
We, i.e. Server WG, consider to create a change proposal to change it back to the behavior of F37, i.e.  lvm.conf option  'use_devicesfile = 0' or alike. But before doing so we want to get to know, *why* the change was made. We guess, someone should have a reason for that. Do you know what the reason is? Or where we can find the information?

Comment 6 Dominik 'Rathann' Mierzejewski 2024-01-18 12:10:01 UTC
The author of that commit, @mcsontos , would be the best person to ask, logically.

Comment 7 Peter Boy 2024-01-18 12:12:03 UTC
OK, thanks!

Comment 8 David Teigland 2024-02-07 19:05:15 UTC
use_devicesfile=1 is meant to be the standard way of using lvm for years now, and it should have been that way in fedora at least a couple years ago.

The devices file feature was introduced to lvm primarily as an "opt in" mechanism for using devices with lvm.  Previously, lvm has always had an "opt out" approach in which devices needed to rejected with the filter to stop lvm from using them.  Over the past several years, it's become increasingly likely that lvm devices attached to a host no longer belong to the host and it's not safe for the host to assume it can use them. e.g. lvm devices are quite likely to belong to a guest VM, and there were many instances of hosts using and corrupting lvm devices that were in use by a guest VM.  Similar problems exist with machines connected to a SAN.

Comment 9 Peter Boy 2024-02-07 20:09:07 UTC
Many thanks for all this information! The arguments are immediately plausible to me. 


I conclude from this

1. 
It is best to operate Fedora Server with the use_devicesfile=1 to be on the save side.


2. 
We should, nevertheless, adapt anaconda (see #2247872 ) to
  a) remove the device file from the OS image. It refers to the machine that creates the image and is most likely incorrect for the machine to be installed.
  b) create a correct device file with first boot


3. 
We need a way to temporarily add a new device using a script. Example is arm-image-installer. It adds a device containing LVM as loop device and later physically via USB or sd slot. There are sections like:

if [ "$(fdisk -l $MEDIA | grep LVM)" != "" ]   ; then
        vgchange -a n "$(pvs 2> /dev/null | grep $MEDIA | awk '{print $2}')" > /dev/null 2>&1
fi

Currently vgchange fails due to the change in LVM configuration. Could you provide us with a hint or example how to do that? I can't see that from what I've found so far. Sorry for my lack of understanding.


4.
For standard LVM administration using the CLI vg-* tools, we need for server documentation a way to temporarily add a device without jeopardizing the system configuration and a way to permanently add a device. For the latter, probably vgimportdevices. 

Please, could you help us with an appropriate sentence/description which I can include in our documentation and in release notes?

Comment 10 David Teigland 2024-02-07 20:45:23 UTC
In terms of temporarily using a device with lvm, there are two options:

1. Add the device to system.devices (lvmdevices --adddev /dev/foo), run lvm commands using that device, then remove the device from system.devices (lvmdevices --deldev /dev/foo.)  This makes sense if there are various and unpredictable lvm commands that might be used in the middle.

2. Give each individual lvm command access to the specific device using "--devices /dev/foo", and avoid needing to use system.devices.  e.g. vgchange -ay --devices /dev/foo.  This might make sense if you have a limited and predictable series of commands that you want to run with a fixed device (or devices, you can add multiple, e.g. --devices /dev/foo,/dev/bar.)  You can pass --devices to all lvm commands, and it's the most optimal way of running commands, because the commands will only look at the named devices, and can't be influenced by any other devices on the system.  Commands that don't include --devices /dev/foo won't see it.

To permamently allow LVM to use a device, run lvmdevices --adddev /dev/foo to add it to system.devices.  If you have an entire existing VG that you want to add to the system, you can run vgimportdevices <vgname> which will add all devices for <vgname> to system.devices.  Or, vgimportdevices -a to do the same for all visible VGs.

The shortest way to communicate that is probably something like:
"To use a device with LVM, run "lvmdevices --adddev device_name" to include that device in /etc/lvm/devices/system.devices.  Or, run vgimportdevices -a to include all devices from all visible VGs."


Note You need to log in before you can comment on or make changes to this bug.