Bug 2014892

Summary:	kernel 5.14.11-200 cannot find file system on Xen 6.2 virtual servers
Product:	[Fedora] Fedora	Reporter:	Lou Duchez <lou>
Component:	kernel	Assignee:	Kernel Maintainer List <kernel-maint>
Status:	CLOSED DUPLICATE	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	34	CC:	acaringi, adscvr, airlied, alciregi, bskeggs, hdegoede, jarodwilson, jeremy, jglisse, jonathan, josef, kernel-maint, lgoncalv, linville, masami256, mchehab, ptalbert, steved, vkuznets
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-10-18 12:01:17 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Lou Duchez 2021-10-17 18:16:51 UTC

I manage Fedora servers on three types of platform: standalone physical servers, virtual servers on VMWare, and virtual servers on Xen 6.2. The latest kernel (kernel-5.14.11-200.fc34.x86_64) works fine on the first two groups of servers. But when I try to reboot my virtual servers on Xen 6.2, I get these errors:

Warning: /dev/fedora/root does not exist
Warning: /dev/fedora/swap does not exist
Warning: /dev/mapper/fedora-root does not exist

Generating "/run/initramfs/rdsosreport.txt"

Entering emergency mode. Exit the shell to continue.

When I boot from the previously-installed version of the kernel (probably last updated 40 days ago), everything works fine again. I didn't see this problem until 5.14.11-200, so apparently it was introduced in 5.14.11-200 or another very recent release. As a benchmark, it didn't happen in 5.13.13-200.

I don't know what further information you would need. I can tell you that the virtual server images are housed on an EMC network storage device. I don't think I can get you an error log since 5.14.11-200 couldn't get at the volume where it would write the error log. I do not know much about the physical configuration of the VMWare system where things are working correctly, so I don't know whether VMWare has to access a NAS of any kind.

Comment 1 Vitaly Kuznetsov 2021-10-18 10:35:13 UTC

This is likely a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=2010058 

TL;DR: it is kernel change in 5.14 which requires a dracut change. Without the change 'xen-blkfront' driver won't be put into initrd.
As a workaround, you can force it there manually ("echo xen-blkfront >> /etc/modules-load.d/xen.conf; dracut --regenerate-all --force")

Comment 2 Lou Duchez 2021-10-18 11:40:51 UTC

This worked, thank you! I have tested ten of my Xen servers and they were able to successfully boot to the new kernel, once I applied that manual fix ("echo xen-blkfront >> /etc/modules-load.d/xen.conf; dracut --regenerate-all --force"). 

Is this viewed as a bug that will be fixed in future releases?

Comment 3 Vitaly Kuznetsov 2021-10-18 12:01:17 UTC

Yes, it will I believe (I'm not a dracut maintainer but apparently there is an unstream fix for this:
https://github.com/dracutdevs/dracut/commit/b292ce72). You can follow BZ#2010058 and I'll close this
one as a duplicate.

*** This bug has been marked as a duplicate of bug 2010058 ***