| Summary: | dracut-fips causes the kernel to panic right after parsing the initrd | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | ben haubeck <bhaubeck> | ||||||
| Component: | kernel | Assignee: | Kernel Maintainer List <kernel-maint> | ||||||
| Status: | CLOSED NOTABUG | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||
| Severity: | unspecified | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 25 | CC: | dracut-maint-list, gansalmon, harald, ichavero, itamar, jonathan, kernel-maint, labbott, madhu.chinakonda, mchehab, sbonazzo, zbyszek | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2016-12-16 16:20:26 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Attachments: |
|
||||||||
what is the output of: # dracut --print-cmdline ? dracut --print-cmdline rd.luks.uuid=luks-ca25bba3-96d8-4664-bd1f-cad046a0af54 rd.lvm.lv=systemvg/rootlv rd.lvm.lv=systemvg/swaplv resume=/dev/mapper/systemvg-swaplv root=/dev/mapper/systemvg-rootlv rootfstype=ext4 rootflags=rw,relatime,seclabel,data=ordered what is the output of:
# egrep -R . /{etc,lib}/dracut.conf*
egrep -R . /{etc,lib}/dracut.conf*
/etc/dracut.conf:# PUT YOUR CONFIG IN separate files
/etc/dracut.conf:# in /etc/dracut.conf.d named "<name>.conf"
/etc/dracut.conf:# SEE man dracut.conf(5) for options
/etc/dracut.conf.d/ben-dracut.conf:add_dracutmodules+=" systemd-bootchart"
/etc/dracut.conf.d/ovirt-dracut.conf:add_dracutmodules+=" ovirtnode multipath fips"
grep: /lib/dracut.conf*: No such file or directory
the file ben-dracut.conf was created today by me, so was not existing as the problem started to occur. ben (In reply to ben haubeck from comment #4) > /etc/dracut.conf.d/ovirt-dracut.conf:add_dracutmodules+=" ovirtnode > multipath fips" I don't know this file, nor do I know "ovirtnode". What is the output of: # rpm -qf /etc/dracut.conf.d/ovirt-dracut.conf And do you really need that rpm? Try removing this config file and recreate the initramfs without it. the file belongs to a package from Fedora21, so I erased the complete package, rebuild the initrd for Kernel 4.8.1-1 (that was 2 mb less then before), but unfortunately the boot with the new initrd fails in the same way. what is the error message? maybe take a photo and attach it here? Created attachment 1210440 [details]
screenshot of the failed boot
screenshot is attached
I tried already rd.break=cmdline but that does not drop me to a shell either.
it has not effects and the error was the same.
(In reply to ben haubeck from comment #9) > Created attachment 1210440 [details] > screenshot of the failed boot > > screenshot is attached > > I tried already rd.break=cmdline but that does not drop me to a shell either. > it has not effects and the error was the same. try "rdinit=/bin/bash" if that doesn't work, then it is the kernel i used "rdinit=/bin/bash" and that leads to a prompt. how can i go further then? and i also removed all Fedora21, 22 and 23 packages on the system and rebuilded the initramfs for 4.8.1-1 but that did not change anything. thanks a lot for your effort! (In reply to ben haubeck from comment #11) > i used > > "rdinit=/bin/bash" > > and that leads to a prompt. > how can i go further then? > > and i also removed all Fedora21, 22 and 23 packages on the system and > rebuilded the initramfs for 4.8.1-1 but that did not change anything. > > thanks a lot for your effort! ok, next step: add only "rd.systemd.unit=emergency.target" to the kernel command line if that works, and you are dropped to a shell type: # systemctl start systemd-udevd # systemctl start systemd-udev-trigger If that works: # systemctl --no-block isolate default.target unfortunately it does not reach the eemergency target. it crashes at the same time with the same message as already attached. (In reply to ben haubeck from comment #13) > unfortunately it does not reach the eemergency target. it crashes at the > same time with the same message as already attached. odd... reassigning to kernel That screenshot isn't particularly helpful unfortunately. It's showing a backtrace that looks like the init process may have taken a signal. Can you get a screenshot of anything before that final trace? any ideas how I can achieve that / slow down it? i tried to videotape it, but was not successfull as my smartphone camera has not sufficient FPS ... try the kernel command line parameters: systemd.confirm_spawn=1 systemd.crash_shell=1 thx Harald. unfortunately it crashes at the same time. i am confused. can I provide the initiramfs for kernel 4.7.6 in a working version (from a notebook from a colleague) and the one that is not working from mine? thx, ben (In reply to ben haubeck from comment #18) > thx Harald. > unfortunately it crashes at the same time. > > i am confused. > can I provide the initiramfs for kernel 4.7.6 in a working version (from a > notebook from a colleague) and the one that is not working from mine? > > thx, ben doesn't compare, if the kernel version is not the same. i have it for exactly the same kernel, but it is a F24 kernel version: i have a running kernel and initramfs for 4.7.6-200.fc24.x86_64 and also a newly build initramfs for this kernel (build after upgrade to F25), that does not boot. ben (In reply to ben haubeck from comment #20) > i have it for exactly the same kernel, but it is a F24 kernel version: > i have a running kernel and initramfs for > > 4.7.6-200.fc24.x86_64 > > and also a newly build initramfs for this kernel (build after upgrade to > F25), that does not boot. > > ben oh, so you have a working and non-working initramfs for the same kernel? care to share those two? does it help, if you uninstall dracut-fips (which you shouldn't have installed in the first place with: # dnf remove dracut-fips and then regenerate the initramfs? is this looking fine for you? or do I have to worry about "dracut: No dracut internal kernel commandline stored in the initramfs" ? root@trouble2> dracut -v --kver 4.8.6-300.fc25.x86_64 --force ~ben/tmp dracut: Executing: /usr/bin/dracut -v --kver 4.8.6-300.fc25.x86_64 --force dracut: *** Including module: bash *** dracut: *** Including module: systemd *** dracut: *** Including module: systemd-bootchart *** dracut: *** Including module: systemd-initrd *** dracut: *** Including module: nss-softokn *** dracut: *** Including module: i18n *** dracut: *** Including module: network *** dracut: *** Including module: ifcfg *** dracut: *** Including module: drm *** dracut: *** Including module: plymouth *** dracut: *** Including module: crypt *** dracut: *** Including module: dm *** dracut: Skipping udev rule: 64-device-mapper.rules dracut: Skipping udev rule: 60-persistent-storage-dm.rules dracut: Skipping udev rule: 55-dm.rules dracut: *** Including module: kernel-modules *** dracut: *** Including module: kernel-network-modules *** dracut: *** Including module: lvm *** dracut: Skipping udev rule: 64-device-mapper.rules dracut: Skipping udev rule: 56-lvm.rules dracut: Skipping udev rule: 60-persistent-storage-lvm.rules dracut: *** Including module: resume *** dracut: *** Including module: rootfs-block *** dracut: *** Including module: terminfo *** dracut: *** Including module: udev-rules *** dracut: Skipping udev rule: 40-redhat.rules dracut: Skipping udev rule: 50-firmware.rules dracut: Skipping udev rule: 50-udev.rules dracut: Skipping udev rule: 91-permissions.rules dracut: Skipping udev rule: 80-drivers-modprobe.rules dracut: *** Including module: biosdevname *** dracut: *** Including module: dracut-systemd *** dracut: *** Including module: usrmount *** dracut: *** Including module: base *** dracut: *** Including module: fs-lib *** dracut: *** Including module: shutdown *** dracut: *** Including modules done *** dracut: *** Installing kernel module dependencies *** dracut: *** Installing kernel module dependencies done *** dracut: *** Resolving executable dependencies *** dracut: *** Resolving executable dependencies done*** dracut: *** Hardlinking files *** dracut: *** Hardlinking files done *** dracut: *** Stripping files *** dracut: *** Stripping files done *** dracut: *** Generating early-microcode cpio image *** dracut: *** Constructing GenuineIntel.bin **** dracut: *** Store current command line parameters *** dracut: Stored kernel commandline: dracut: No dracut internal kernel commandline stored in the initramfs dracut: *** Creating image file '/boot/initramfs-4.8.6-300.fc25.x86_64.img' *** dracut: *** Creating initramfs image file '/boot/initramfs-4.8.6-300.fc25.x86_64.img' done *** root@trouble2> (In reply to ben haubeck from comment #24) > is this looking fine for you? > or do I have to worry about > > "dracut: No dracut internal kernel commandline stored in the initramfs" > > ? > That is fine.. Everything without "warning" or "error" is fine. you are great!!
obviously it was the darcut-fips package, that caused the whole mess.
for whatever reason I installed that.
root@trouble2> dnf history info 200 /home/ben/tmp
Transaction ID : 200
Begin time : Tue Jun 30 23:11:23 2015
Begin rpmdb : 2347:eea9d3c591ce9e7e75392437d49f55665df7b9c8
End time : (0 seconds)
End rpmdb : 2349:05a5e82232cabe44ca1ec94ff502ffccc3ca9219
User : Benjamin Haubeck <ben>
Return-Code : Success
Command Line : install -y dracut-fips.x86_64
Transaction performed with:
Upgraded dnf-1.0.1-2.fc22.noarch @updates/22
Upgraded rpm-4.12.0.1-11.fc22.x86_64 @updates/22
Packages Altered:
Install dracut-fips-041-14.fc22.x86_64 @updates/22
Install hmaccalc-0.9.14-1.fc22.x86_64 @fedora/22
root@trouble2>
interesting, that it sits there in the corner for over a year and over 2 releases without doing any harm ... ?!
thx, for investigating and helping me out.
ben
*** Bug 1398537 has been marked as a duplicate of this bug. *** Closing this bug now and updating the title to indicate the problem. |
Created attachment 1210017 [details] lsinitrd of broken initrd Description of problem: I have upgraded via dnf to Fedora 25 Beta and installed the kernel 4.8.1 and rebuilded the initrd for Kernel 4.8.1-1 and for 4.7.6-200. My rootlv is encrypted via LUKS on an LVM volume. Every initrd I am building now for both kernel i have (4.8.1-1 and for 4.7.6-200) is not able to boot my machine. I copied the initramfs for the kernel kernel-4.7.6-200.fc24.x86_64 from a colleagues machine over to mine to be able to boot it again. I did not change anything in my grub2.cfg Version-Release number of selected component (if applicable): dracut-044-77.fc25.x86_64 dracut-fips-044-77.fc25.x86_64 kernel-4.8.1-1.fc25.x86_64 kernel-4.7.6-200.fc24.x86_64 kernel-modules-4.7.6-200.fc24.x86_64 kernel-tools-libs-4.8.1-1.fc25.x86_64 libreport-plugin-kerneloops-2.8.0-1.fc25.x86_64 kernel-modules-extra-4.8.1-1.fc25.x86_64 dracut-network-044-77.fc25.x86_64 kernel-core-4.8.1-1.fc25.x86_64 kernel-tools-4.8.1-1.fc25.x86_64 dracut-config-rescue-044-77.fc25.x86_64 kernel-headers-4.8.1-1.fc25.x86_64 kernel-core-4.7.6-200.fc24.x86_64 abrt-addon-kerneloops-2.9.0-1.fc25.x86_64 kernel-debug-devel-4.7.6-200.fc24.x86_64 kernel-modules-4.8.1-1.fc25.x86_64 kernel-modules-extra-4.7.6-200.fc24.x86_64 kernel-debug-devel-4.8.1-1.fc25.x86_64 fedup-dracut-plymouth-0.9.2-3.fc23.noarch How reproducible: everytime Steps to Reproduce: 1. upgrade to Fedora 25 (maybe only on a system with LUKS rootlv?) 2. use dracut (-f) to (re)build the initrd 3. see kernel panic right after grub finished his tasks Actual results: not working initrd Expected results: working initrd Additional info: i attach the output of lsinitrd of one of the broken initrd my grub2.cfg: # # DO NOT EDIT THIS FILE # # It is automatically generated by grub2-mkconfig using templates # from /etc/grub.d and settings from /etc/default/grub # ### BEGIN /etc/grub.d/00_header ### set pager=1 if [ -s $prefix/grubenv ]; then load_env fi if [ "${next_entry}" ] ; then set default="${next_entry}" set next_entry= save_env next_entry set boot_once=true else set default="${saved_entry}" fi if [ x"${feature_menuentry_id}" = xy ]; then menuentry_id_option="--id" else menuentry_id_option="" fi export menuentry_id_option if [ "${prev_saved_entry}" ]; then set saved_entry="${prev_saved_entry}" save_env saved_entry set prev_saved_entry= save_env prev_saved_entry set boot_once=true fi function savedefault { if [ -z "${boot_once}" ]; then saved_entry="${chosen}" save_env saved_entry fi } function load_video { if [ x$feature_all_video_module = xy ]; then insmod all_video else insmod efi_gop insmod efi_uga insmod ieee1275_fb insmod vbe insmod vga insmod video_bochs insmod video_cirrus fi } terminal_output console if [ x$feature_timeout_style = xy ] ; then set timeout_style=menu set timeout=5 # Fallback normal timeout code in case the timeout_style feature is # unavailable. else set timeout=5 fi ### END /etc/grub.d/00_header ### ### BEGIN /etc/grub.d/10_linux ### menuentry 'Fedora (4.8.1-1.fc25.x86_64) 25 (Twenty Five)' --class fedora --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-0-rescue-b6ae0e91b1664e8ebd5699aaa01894a6-advanced-e5758c7d-e077-4ece-8043-c59604836013' { load_video insmod gzio insmod part_msdos insmod ext2 set root='hd0,msdos1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 --hint='hd0,msdos1' 3c8cf32a-1217-4a46-9f94-46ae0b40c27d else search --no-floppy --fs-uuid --set=root 3c8cf32a-1217-4a46-9f94-46ae0b40c27d fi linux16 /vmlinuz-4.8.1-1.fc25.x86_64 root=/dev/mapper/systemvg-rootlv ro rd.lvm.lv=systemvg/rootlv rd.lvm.lv=systemvg/swaplv rd.luks.uuid=luks-ca25bba3-96d8-4664-bd1f-cad046a0af54 LANG=en_GB.UTF-8 initrd16 /initramfs-4.8.1-1.fc25.x86_64.img } menuentry 'Fedora (4.7.6-200.fc24.x86_64) 24 (Twenty Four)' --class fedora --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-0-rescue-b6ae0e91b1664e8ebd5699aaa01894a6-advanced-e5758c7d-e077-4ece-8043-c59604836013' { load_video insmod gzio insmod part_msdos insmod ext2 set root='hd0,msdos1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 --hint='hd0,msdos1' 3c8cf32a-1217-4a46-9f94-46ae0b40c27d else search --no-floppy --fs-uuid --set=root 3c8cf32a-1217-4a46-9f94-46ae0b40c27d fi linux16 /vmlinuz-4.7.6-200.fc24.x86_64 root=/dev/mapper/systemvg-rootlv ro rd.lvm.lv=systemvg/rootlv rd.lvm.lv=systemvg/swaplv rd.luks.uuid=luks-ca25bba3-96d8-4664-bd1f-cad046a0af54 LANG=en_GB.UTF-8 initrd16 /initramfs-4.7.6-200.fc24.x86_64.img } menuentry 'Fedora, with Linux 0-rescue-b6ae0e91b1664e8ebd5699aaa01894a6' --class fedora --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-0-rescue-b6ae0e91b1664e8ebd5699aaa01894a6-advanced-e5758c7d-e077-4ece-8043-c59604836013' { load_video insmod gzio insmod part_msdos insmod ext2 set root='hd0,msdos1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 --hint='hd0,msdos1' 3c8cf32a-1217-4a46-9f94-46ae0b40c27d else search --no-floppy --fs-uuid --set=root 3c8cf32a-1217-4a46-9f94-46ae0b40c27d fi linux16 /vmlinuz-0-rescue-b6ae0e91b1664e8ebd5699aaa01894a6 root=/dev/mapper/systemvg-rootlv ro rd.lvm.lv=systemvg/rootlv rd.lvm.lv=systemvg/swaplv rd.luks.uuid=luks-ca25bba3-96d8-4664-bd1f-cad046a0af54 initrd16 /initramfs-0-rescue-b6ae0e91b1664e8ebd5699aaa01894a6.img } ### END /etc/grub.d/10_linux ### ### BEGIN /etc/grub.d/20_linux_xen ### ### END /etc/grub.d/20_linux_xen ### ### BEGIN /etc/grub.d/20_ppc_terminfo ### ### END /etc/grub.d/20_ppc_terminfo ### ### BEGIN /etc/grub.d/30_os-prober ### ### END /etc/grub.d/30_os-prober ### ### BEGIN /etc/grub.d/40_custom ### # This file provides an easy way to add custom menu entries. Simply type the # menu entries you want to add after this comment. Be careful not to change # the 'exec tail' line above. ### END /etc/grub.d/40_custom ### ### BEGIN /etc/grub.d/41_custom ### if [ -f ${config_directory}/custom.cfg ]; then source ${config_directory}/custom.cfg elif [ -z "${config_directory}" -a -f $prefix/custom.cfg ]; then source $prefix/custom.cfg; fi ### END /etc/grub.d/41_custom ###