Bug 1644869

Summary: dracut failes to find LVM volumes
Product: [Fedora] Fedora Reporter: Ben Cotton <bcotton>
Component: dmraidAssignee: Heinz Mauelshagen <heinzm>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 29CC: agk, bmr, cfeist, dracut-maint-list, harald, heinzm, jbrassow, jonathan, kzak, lvm-team, mcsontos, prajnoha, prockai, teigland, turchi, zbyszek, zkabelac
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-11-27 23:26:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
dracut report
none
Grub config
none
lvdisplay output
none
vgdisplay output
none
pvdisplay output
none
dmesg output
none
udev blkdid output
none
blkid output
none
dmsetup output
none
/etc/fstab contents
none
/proc/cmdline none

Description Ben Cotton 2018-10-31 19:03:14 UTC
Created attachment 1499561 [details]
dracut report

Description of problem:
When booting this system, dracut times out trying to initialize storage. This works when using a Fedora 26 kernel (and F29 everything else), but fails on newer kernels. Filing this against dracut initially since that's where the error messages come from, but it may be a kernel issue.

Various outputs attached.

Comment 1 Ben Cotton 2018-10-31 19:04:05 UTC
Created attachment 1499562 [details]
Grub config

Comment 2 Ben Cotton 2018-10-31 19:04:31 UTC
Created attachment 1499563 [details]
lvdisplay output

Comment 3 Ben Cotton 2018-10-31 19:04:54 UTC
Created attachment 1499564 [details]
vgdisplay output

Comment 4 Ben Cotton 2018-10-31 19:05:20 UTC
Created attachment 1499565 [details]
pvdisplay output

Comment 5 Ben Cotton 2018-10-31 19:06:11 UTC
Created attachment 1499566 [details]
dmesg output

Comment 6 Ben Cotton 2018-10-31 19:06:55 UTC
Created attachment 1499567 [details]
udev blkdid output

Comment 7 Ben Cotton 2018-10-31 19:07:21 UTC
Created attachment 1499568 [details]
blkid output

Comment 8 Ben Cotton 2018-10-31 19:07:41 UTC
Created attachment 1499569 [details]
dmsetup output

Comment 9 Ben Cotton 2018-10-31 19:09:10 UTC
Created attachment 1499570 [details]
/etc/fstab contents

Comment 10 Ben Cotton 2018-10-31 19:09:40 UTC
Created attachment 1499571 [details]
/proc/cmdline

Comment 11 Harald Hoyer 2018-11-12 08:30:21 UTC
To recap:

Desired state:

fedora_normal-swap (253:4)
 └─pdc_gfeajffc2 (253:2)
    └─pdc_gfeajffc (253:0)
       ├─ (8:16)
       └─ (8:0)
fedora_normal-root (253:3)
 └─pdc_gfeajffc2 (253:2)
    └─pdc_gfeajffc (253:0)
       ├─ (8:16)
       └─ (8:0)

Dracut run:

[    3.394156] normal.lan dracut-initqueue[433]: Scanning for dmraid devices pdc_gfeajffc
[    3.554144] normal.lan dracut-initqueue[433]: Found dmraid sets:
[    3.554641] normal.lan dracut-initqueue[433]: pdc_gfeajffc
[    3.554917] normal.lan dracut-initqueue[433]: Activating pdc_gfeajffc
[    3.580069] normal.lan dracut-initqueue[433]: RAID set "pdc_gfeajffc" was activated
[    3.580146] normal.lan dracut-initqueue[433]: RAID set "pdc_gfeajffc" was not activated

So, pdc_gfeajffc seems to be in a strange state...

+ lvm pvdisplay
  /dev/sdf: open failed: No medium found
  Read-only locking type set. Write locks are prohibited.
  Recovery of standalone physical volumes failed.
  Cannot process standalone physical volumes
+ lvm vgdisplay
  /dev/sdf: open failed: No medium found
+ lvm lvdisplay
  /dev/sdf: open failed: No medium found
+ command -v dmsetup
+ dmsetup ls --tree
pdc_gfeajffc (253:0)
 |- (8:16)
 `- (8:0)


Reassigning to dmraid...

Comment 12 David Teigland 2018-11-12 16:17:16 UTC
lvm pvdisplay
  /dev/sdf: open failed: No medium found
  Read-only locking type set. Write locks are prohibited.
  Recovery of standalone physical volumes failed.
  Cannot process standalone physical volumes

This fix might be needed, I'm not sure if this is in the lvm version being used
https://sourceware.org/git/?p=lvm2.git;a=commit;h=0aeca60aaa65bde99115ec7ca92a93a885d3afb1

Someone else on the mailing list reported these no medium found errors, but last I checked there was nothing in lvm printing those errors, so I started to think they might be coming from blkid.

Comment 13 Ben Cotton 2018-11-12 17:13:39 UTC
Given the date of the broken commit, I'm not sure it applies (I've been having this problem for a while (possibly since sometime in F27?) and only recently reported it).

One thing I didn't mention initially is that the PV pdc_gfeajffc2 is a hardware RAID 1, except that it's fakeraid. I wonder if that's causing part of the confusion during the boot process. Maybe something changed in the stack since F26 that changes how this gets handled at boot? I'm still leaning toward it ultimately being a kernel issue, since changing the kernel is what makes a difference, but maybe it's in how the initrd is generated or something else entirely.

Comment 14 Harald Hoyer 2018-11-16 08:44:45 UTC
Have you tried mdraid instead of dmraid?

Comment 15 Ben Cotton 2018-11-16 15:51:16 UTC
I have not tried using mdraid yet. I didn't want to make any potentially-destructive changes since I never got around to setting up config management on this host.

Comment 16 Gabriele Turchi 2019-01-01 09:25:14 UTC
I have a similar problem: installed a F29 over a F20 created volume group via live image with no problem, at boot the system stops with an lvm error (Recovery of volume group failed). It's a laptop, so no strange raid/md/dm configuration applies.

Comment 17 Heinz Mauelshagen 2019-09-17 17:31:36 UTC
(In reply to Harald Hoyer from comment #14)
> Have you tried mdraid instead of dmraid?

mdraid, i.e. mdadm does not support the 'Promise FastTrack' format aka 'pdc',
it supports SNIS DDF, Intel IMSM and a few native MD metadata formats
versioned 0.90, 1.0, 1.1 and 1.2.

It can be deployed directly on the disks thus avoiding Promise FakeRAID altogether.
This would mean OS reinstallation though.

Also mind, 'pdc' is no hardware raid at all, just the SATA interfaces and a BIOS extension to allow
to configure e.g. raid1 on the attached disks and booting off it which requires dmraid to activate
such BIOS formated raid in Linux.

Comment 18 Ben Cotton 2019-10-31 18:50:06 UTC
This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 19 Ben Cotton 2019-11-27 23:26:04 UTC
Fedora 29 changed to end-of-life (EOL) status on 2019-11-26. Fedora 29 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.