Bug 1672817
| Summary: | kdump fails to generate vmcore if dump target is nfs ,ssh or localfilesystem when fips is enabled (fips=1) | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Steve Barcomb <sbarcomb> | ||||
| Component: | dracut | Assignee: | Lukáš Nykrýn <lnykryn> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Release Test Team <release-test-team-automation> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 7.6 | CC: | dracut-maint-list, jstodola, kasong, kdsouza, lnykryn, nhorman, ruyang, sbarcomb, xiawu | ||||
| Target Milestone: | beta | ||||||
| Target Release: | --- | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | dracut-033-560.el7 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1688608 (view as bug list) | Environment: | |||||
| Last Closed: | 2019-08-06 13:13:38 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1688608, 1721114 | ||||||
| Attachments: |
|
||||||
|
Description
Steve Barcomb
2019-02-06 00:55:48 UTC
Can you test see if this is also reproducible in RHEL8? Hey Dave, There's not a lot of good documentation about kdump and FIPS, but should the boot=UUID=<the UUID> be needed in /etc/sysconfig/kdump? I tested this on 2 FIPS compliant virtual machines after adding the boot UUIDS in that manner to the KDUMP_COMMANDLINE_APPEND= and got a little further: [ 2.037168] mce: Unable to init device /dev/mcelog (rc: -5) [ 6.958169] irq 11: nobody cared (try booting with the "irqpoll" option) [ 6.959177] handlers: [ 6.959549] [<ffffffff814f9e30>] usb_hcd_irq [ 6.960256] [<ffffffff814f9e30>] usb_hcd_irq [ 6.960963] Disabling IRQ #11 [ 7.841521] dracut-pre-trigger[221]: mount: unknown filesystem type 'xfs' [ 11.032478] dracut-pre-pivot[2156]: mkdir: cannot create directory '/boot': File exists [ 11.038834] dracut-pre-pivot[2156]: mount: unknown filesystem type 'xfs' [ 11.213259] dracut-pre-pivot[2156]: modprobe: FATAL: Module sha1 not found. [ 11.366727] dracut-pre-pivot[2156]: modprobe: FATAL: Module sha256 not found. [ 11.694913] dracut: FATAL: FIPS integrity test failed [ 11.695485] dracut: Refusing to continue [ 11.605383] dracut-pre-pivot[2156]: Warning: /boot/.vmlinuz-3.10.0-957.el7.x86_64.hmac does not exist [ 11.775222] System halted. Obviously the hmac file exists in boot, but does not seem available to the crash environment. It seems we had something similar in bz 909402 I can test this on RHEL8, but it might take some time to do. >> Does adding "-a fips" in dracut_args work?
see /etc/kdump.conf: dracut_args
No, still fails with the same error.
My configuration:
# kdump-config --show
path /var/crash
core_collector makedumpfile -l --message-level 1 -d 31
ext4 UUID=e726e912-a045-4028-b9fd-efdb6da76fc6
dracut_args -a fips
# cat /etc/fstab | grep -i boot
UUID=cad41ff6-0954-4e2b-8f45-91880f94590c /boot xfs defaults 0 0
# df -h | grep -w test
/dev/sda 2.0G 6.1M 1.8G 1% /test
# lsinitrd /boot/initramfs-$(uname -r)kdump.img /etc/fstab
/dev/disk/by-uuid/e726e912-a045-4028-b9fd-efdb6da76fc6 /kdumproot//test ext4 defaults 0 2
XFS module is not inserted in kdump initramfs ( for /boot)
# lsinitrd /boot/initramfs-$(uname -r)kdump.img | grep -i xfs | wc -l
0
# lsinitrd /boot/initramfs-$(uname -r)kdump.img | grep -i fips
Arguments: --hostonly --hostonly-cmdline --hostonly-i18n --hostonly-mode 'strict' -o 'plymouth dash resume ifcfg' --mount '/dev/disk/by-uuid/e726e912-a045-4028-b9fd-efdb6da76fc6 /kdumproot//test ext4 defaults' -a 'fips' --no-hostonly-default-device -f
fips
-rw-r--r-- 1 root root 441 Mar 5 06:37 etc/fipsmodules
-rw-r--r-- 1 root root 1907 Mar 5 06:37 etc/modprobe.d/fips.conf
-rw-r--r-- 1 root root 0 Sep 27 09:47 etc/system-fips
-rwxr-xr-x 1 root root 15736 Feb 21 2017 usr/bin/fipscheck
drwxr-xr-x 2 root root 0 Mar 5 06:37 usr/lib64/fipscheck
-rw-r--r-- 1 root root 65 Feb 21 2017 usr/lib64/fipscheck/fipscheck.hmac
-rw-r--r-- 1 root root 65 Feb 21 2017 usr/lib64/fipscheck/libfipscheck.so.1.2.1.hmac
lrwxrwxrwx 1 root root 26 Mar 5 06:37 usr/lib64/fipscheck/libfipscheck.so.1.hmac -> libfipscheck.so.1.2.1.hmac
-rwxr-xr-x 1 root root 11344 Feb 21 2017 usr/lib64/libfipscheck.so.1.2.1
lrwxrwxrwx 1 root root 21 Mar 5 06:37 usr/lib64/libfipscheck.so.1 -> libfipscheck.so.1.2.1
-rwxr-xr-x 1 root root 354 Sep 12 2013 usr/lib/dracut/hooks/pre-pivot/01-fips-noboot.sh
-rwxr-xr-x 1 root root 375 Sep 12 2013 usr/lib/dracut/hooks/pre-trigger/01-fips-boot.sh
-rwxr-xr-x 1 root root 4615 Sep 27 09:46 usr/sbin/fips.sh
After crashing:
# echo c > /proc/sysrq-trigger
[ 2207.776394] RIP [<ffffffffb7c62276>] sysrq_handle_crash+0x16/0x20
[ 2207.778615] RSP <ffff92a8b9babe58>
[ 2207.780401] CR2: 0000000000000000
[ 0.534364] do_IRQ: 0.98 No irq handler for vector (irq -1)
[ 0.703056] mce: Unable to init device /dev/mcelog (rc: -5)
[ 1.661599] dracut-pre-trigger[86]: mount: unknown filesystem type 'xfs'
[ 3.039196] systemd-fsck[1905]: /dev/sda: recovering journal
[ 3.059937] systemd-fsck[1905]: /dev/sda: clean, 13/131072 files, 26158/524288 blocks
[ 3.184669] dracut-pre-pivot[1932]: mkdir: cannot create directory '/boot': File exists
[ 3.194212] dracut-pre-pivot[1932]: mount: unknown filesystem type 'xfs' <============
[ 3.427619] dracut-pre-pivot[1932]: modprobe: FATAL: Module sha1 not found.
[ 3.624749] dracut-pre-pivot[1932]: modprobe: FATAL: Module sha256 not found.
[ 4.082923] dracut: FATAL: FIPS integrity test failed
[ 3.940700] dracut-pre-pivot[ 4.085520] dracut: Refusing to continue
[1932]: Warning: /boot/.vmlinuz-3.10.0-957.5.1.el7.x86_64.hmac does not exist
[ 4.146427] System halted.
We might need to fix the issue in dracut package.
Not sure if the patch should be for dracut-fips?
We should detect fips and add fips dracut module in kexec-tools, and according to your test , also need a fix in dracut for the missing kernel module. I noticed below: Warning: /boot/.vmlinuz-3.10.0-957.5.1.el7.x86_64.hmac Probably fips depends on this? Kairui, can you take a look at this? Thanks! Seems the dracut fips module is added, but mouting /boot/ failed because of lacking the fs driver, shouldn't the driver be included automatically. Lukas, any idea about this? To be honest this question made me a bit uncertain, but if I am not mistaken (and man bootup says the same thing), the kernel is responsible for loading initramdisk. That means that in the normal boot kernel has to read /boot and so it can't have the driver for that partition compiled as a module. Hmm, boot loader should be able to read /boot and load kernel and initramdisk, then kernel just boot up and jump into init root fs. Reading the docs below https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/security_guide/chap-federal_standards_and_regulations User add fips=1 and boot= in kernel cmdline to enable fips, and dracut mount /boot according to boot= provided infomation. It just works in normal boot maybe because of /boot and root use same file system, and we have root= in cmdline and root is mandatory so dracut just packs the fs module in initrd. But for kdump we do not need root filesystem in case ssh/nfs dump, thus the module is not packed in. If this is true probably dracut can add the related fs module in 01fips dracut module because we have to mount /boot in 01fips. Moving this to dracut, I've submitted a pr to fix it: https://github.com/dracutdevs/dracut/pull/553/commits/8b6b3efab39a0ccbe918c92a208b86c06680f7f0 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2289 |