Bug 2271185 - boot fails: cancel waiting for multipath siblings of nvme0n1
Summary: boot fails: cancel waiting for multipath siblings of nvme0n1
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: dracut
Version: 39
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: dracut-maint-list
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2024-03-23 13:21 UTC by Henning
Modified: 2024-11-28 11:51 UTC (History)
20 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2024-11-28 11:51:43 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
anaconda fails to boot (1.67 MB, image/jpeg)
2024-03-23 13:21 UTC, Henning
no flags Details
boot error 2 (1.27 MB, image/jpeg)
2024-03-23 13:22 UTC, Henning
no flags Details

Description Henning 2024-03-23 13:21:00 UTC
Created attachment 2023215 [details]
anaconda fails to boot

Description of problem:
Downloaded the latest Kinoite 39 iso, flashed to USB stick using Impression Flatpak (using udisks2). 

Booting from USB on a Clevo NV41MZ with 11th gen intel i7, only a Crucial P5 NVME SSD in the correct slot, nothing else.

It always hangs at this step and then errors out with dracut timeout (see photos attached).

the NVME is normally shown and usable from Fedora Kinoite 40 on an external Sata SSD, I could read and write data etc.

I tried
- deleting all partitions
- reflashing the USB stick
- recreating the GPT partition table
- changing some recommended things I dont understand with nvme-cli

Discussion: https://discussion.fedoraproject.org/t/109220

Version-Release number of selected component (if applicable):
Latest Kinoite 39 iso

How reproducible:
Always

Steps to Reproduce:
1. Boot latest Kinoite ISO flashed to USB from USB with nvme without any partitions

Actual results:
Prevents Anaconda from booting while working normally on a running system

Expected results:
Dont know what multipart is, but this should not happen, and even if not prevent Anaconda from booting.

Additional info:
See Discussion and images

Comment 1 Henning 2024-03-23 13:22:03 UTC
Created attachment 2023216 [details]
boot error 2

Comment 2 Henning 2024-04-14 11:56:34 UTC
This is a pretty critical issue as it prevents installation. I needed to dd an old install, and now I get filesystem write issues and not unallocated space, while the space should have 800GB free.

There seems to be a broken partition table, a reinstall would fix that but it is not possible.

Comment 3 Tomáš Bžatek 2024-04-16 13:46:15 UTC
For the sake of completness, could you post `nvme list -v -o json` here please?

This systemd unit is created on-the-fly by 62-multipath.rules, coming from device-mapper-multipath. Possibly similar to bug 1916168#c38. Reassigning.

Comment 4 Ben Marzinski 2024-04-16 16:22:35 UTC
To verify whether multipath is responsible for this issue, when you boot, could you add

rd.multipath=off

to the kernel boot parameters in the grub menu? This will completely disable multipath. If the hang still exists then something else is causing it, and those multipath messages just happen to be the last thing you see.

Comment 5 Henning 2024-07-11 23:31:28 UTC
Hi, sorry for the late reply. I added the argument at the end of the probably correct like (included "quiet") in the grub config.

Didnt change anything.

This does not happen on F40 yet, I just get another error.

https://pagure.io/fedora-kde/SIG/issue/544

Comment 6 Henning 2024-07-11 23:31:50 UTC
*line

Comment 7 Ben Marzinski 2024-07-12 15:58:14 UTC
But you no longer see the "cancel waiting for multipath siblings ..." line, correct? You shouldn't if multipath has been disabled. The issue here is that message happens on a timeout, which can occur even if booting is stuck on something else. This means that you might see that as the last message, even though it has nothing to do with why the system is stuck.  Assuming that it is no longer present, the multipath should have been successfully disabled, and it is clearly unrelated to you booting issue. If so, this issue should probably be assigned to dracut, since the problem is happening in the initramfs.

Comment 8 Henning 2024-07-14 19:26:30 UTC
I still get the message

"Started cancel-multipath-wait-nvme0n1.sevice - cancel waiting for multipath siblings of nvme0n1."

1. In GRUB hover over "Fedora 39" press "e"
2. In the boot config, I added rd.multipath=off to the third line, at the end. The line starts with something pxe. Adding it to the 4th line too resulted in an error
3. Press f10 to boot

This did not change anything it takes a while on basic.target, then the message appears, then I get the spammed output.

Comment 9 Henning 2024-07-14 19:32:58 UTC
Yes evwn though I only get the multipath error on F39, this seems to be the same issue I reported recently and it also occurs on F40.

Comment 11 Ben Marzinski 2024-07-19 20:33:42 UTC
This doesn't happen on F40 because starting with that release, multipath automatically blacklists all nvme devices if native nvme multipathing (the default) is detected. So this is pretty definitively not a multipath issue. I'm reassigning this to dracut since the problem is happening in the initramfs, but I'm not at all sure that the problem lies there. It's possible that the issue is with the Kinoite images themselves. Have you tried this with Silverblue instead to see if that works?

Comment 12 Henning 2024-07-19 22:43:48 UTC
Was this fix already backported to F39? Then this is fixed.

The other issue is on F39 and F40 and the other boot issues.

https://bugzilla.redhat.com/show_bug.cgi?id=2297460

Comment 13 Ben Marzinski 2024-07-22 15:32:32 UTC
No, and I have yet to see any evidence that this is actually responsible for the boot issue. Like I mentioned before, that multipath message is triggered by a timeout. If the boot has hung, when the timeout occurs that message will get printed even if it has nothing to do with the hang.  Because of that, this is often the last message that appears during boot hangs, regardless of the cause. Since the hang is still happening in f40 when multipath is completely ignoring those devices, it doesn't appear to be the cause.

Comment 14 Aoife Moloney 2024-11-13 12:18:26 UTC
This message is a reminder that Fedora Linux 39 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 39 on 2024-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '39'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 39 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 15 Aoife Moloney 2024-11-28 11:51:43 UTC
Fedora Linux 39 entered end-of-life (EOL) status on 2024-11-26.

Fedora Linux 39 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of Fedora Linux
please feel free to reopen this bug against that version. Note that the version
field may be hidden. Click the "Show advanced fields" button if you do not see
the version field.

If you are unable to reopen this bug, please file a new report against an
active release.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.