Bug 2338735

Summary: Fedora 40 upgrade fails resulting in unbootable system
Product: [Fedora] Fedora Reporter: ochal
Component: lvm2Assignee: LVM and device-mapper development team <lvm-team>
Status: NEW --- QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 42CC: agk, anprice, bmarzins, bmr, cfeist, kzak, lvm-team, mcsontos, prajnoha, zkabelac
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description ochal 2025-01-18 15:43:29 UTC
I have a system where /home is on a raid 5 volume that is not detected at boot and not activated.

I can install fedora 40 and update packages just fine if I blacklist Lvm2 and stay on version 2.03.23-1.fc40 but updating lvm2 kills my system

Reproducible: Always

Steps to Reproduce:
update lvm2 to 2.03.25-fc40
reboot
Actual Results:  
no complete boot

Expected Results:  
Boot to desktop

lvmconfig 
  WARNING: Running as a non-root user. Functionality may be unavailable.
config {
}
local {
}
dmeventd {
}
activation {
}
global {
}
shell {
}
backup {
}
log {
}
allocation {
}
devices {
	scan_lvs=1
}

Comment 1 ochal 2025-01-18 15:45:51 UTC
dnf config:

# see `man dnf.conf` for defaults and possible options

[main]
gpgcheck=True
installonly_limit=3
clean_requirements_on_remove=True
best=False
skip_if_unavailable=True
#exclude=lvm2*

Comment 2 ochal 2025-01-19 15:24:08 UTC
I managed to work around most of my issue's, i suspect most stem from a lack of devicefiles, i'll investigate the failure to mount to mount the luks volume next weekend but it's probably unrelaated to lvm proper

Comment 3 ochal 2025-02-02 10:45:23 UTC
did some more tests updating thesew packages render my /home unmountable'

Total download size: 3.0 M
Is this ok [y/N]: y
Downloading Packages:
(1/7): device-mapper-event-1.02.199-1.fc40.x86_ 212 kB/s |  33 kB     00:00    
(2/7): device-mapper-event-libs-1.02.199-1.fc40 182 kB/s |  31 kB     00:00    
(3/7): device-mapper-1.02.199-1.fc40.x86_64.rpm 701 kB/s | 136 kB     00:00    
(4/7): device-mapper-libs-1.02.199-1.fc40.x86_6 1.9 MB/s | 176 kB     00:00    
(5/7): lvm2-2.03.25-1.fc40.x86_64.rpm           6.0 MB/s | 1.5 MB     00:00    
(6/7): lvm2-dbusd-2.03.25-1.fc40.noarch.rpm     687 kB/s | 168 kB     00:00    
(7/7): lvm2-libs-2.03.25-1.fc40.x86_64.rpm      1.5 MB/s | 998 kB     00:00    
--------------------------------------------------------------------------------
Total                                           2.9 MB/s | 3.0 MB     00:01     

if i reboot after these updates i'm dropped into emergency mode and asked for my root password
doing vgchange -ay proerly activates my volume group and i can continue my boot process

If you require more information, let me know

Comment 4 Marian Csontos 2025-02-03 12:49:14 UTC
Hi, could you please upload/post `lvmdump -su`

Comment 5 ochal 2025-02-07 19:39:32 UTC
The lvmdump is here: https://we.tl/t-eI6GFYqE9J

Comment 6 Zdenek Kabelac 2025-02-18 15:57:46 UTC
The thing to check - is the use of 'devicesfile'

If such file '/etc/lvm/devices/system.devices'
(and lvm.conf   use_devicesfile  is 1)

then this file needs to have listed devices used by system.

Unsure how your upgrade was made - there was the idea if the 'file' does NOT exist and you have lvm.conf  which now defaults to 1 for Fedora builds,  lvm2 will still go with 'filters'.

However if the upgrade process has created  'devicesfile'  and BUT  has NOT imported all currently visible  PVs  (with lvmdevices/vgimportdevices) - then you may possibly end with a system which will not see a PV  ( as it's not enlisted in  devicesfile).

Number of solutions -  remove devicesfile and set use_devicesfile=0 
Or simply import visible PV to your devicesfile.

Comment 7 ochal 2025-02-23 09:35:06 UTC
(In reply to Zdenek Kabelac from comment #6)
> The thing to check - is the use of 'devicesfile'
> 
> If such file '/etc/lvm/devices/system.devices'
> (and lvm.conf   use_devicesfile  is 1)
> 
> then this file needs to have listed devices used by system.
> 
> Unsure how your upgrade was made - there was the idea if the 'file' does NOT
> exist and you have lvm.conf  which now defaults to 1 for Fedora builds, 
> lvm2 will still go with 'filters'.
> 
> However if the upgrade process has created  'devicesfile'  and BUT  has NOT
> imported all currently visible  PVs  (with lvmdevices/vgimportdevices) -
> then you may possibly end with a system which will not see a PV  ( as it's
> not enlisted in  devicesfile).
> 
> Number of solutions -  remove devicesfile and set use_devicesfile=0 
> Or simply import visible PV to your devicesfile.

This tracks with what I observed with the f40 installer I did notice this in lvm.conf:
 # Allow LVM LVs to be used as PVs. When enabled, LVM commands will
        # scan active LVs to look for other PVs. Caution is required to
        # avoid using PVs that belong to guest images stored on LVs.
        # When enabled, the LVs scanned should be restricted using the
        # devices file or the filter. This option does not enable autoactivation
        # of layered VGs, which requires editing LVM udev rules (see LVM_PVSCAN_ON_LVS.)
        # This configuration option has an automatic default value.
        # scan_lvs = 0


How do i work with layered VG's? I recall having done that years ago to work with with a mirroreed cache? googling LVM_PVSCAN_ON_LVS didn't provide more information

Comment 8 Aoife Moloney 2025-02-26 13:24:41 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 42 development cycle.
Changing version to 42.