Bug 1672817

Can you test see if this is also reproducible in RHEL8?

Hey Dave,
There's not a lot of good documentation about kdump and FIPS, but should the boot=UUID=<the UUID> be needed in /etc/sysconfig/kdump?  I tested this on 2 FIPS compliant virtual machines after adding the boot UUIDS in that manner to the KDUMP_COMMANDLINE_APPEND= and got a little further:

[    2.037168] mce: Unable to init device /dev/mcelog (rc: -5)
[    6.958169] irq 11: nobody cared (try booting with the "irqpoll" option)
[    6.959177] handlers:
[    6.959549] [<ffffffff814f9e30>] usb_hcd_irq
[    6.960256] [<ffffffff814f9e30>] usb_hcd_irq
[    6.960963] Disabling IRQ #11
[    7.841521] dracut-pre-trigger[221]: mount: unknown filesystem type 'xfs'
[   11.032478] dracut-pre-pivot[2156]: mkdir: cannot create directory '/boot': File exists
[   11.038834] dracut-pre-pivot[2156]: mount: unknown filesystem type 'xfs'
[   11.213259] dracut-pre-pivot[2156]: modprobe: FATAL: Module sha1 not found.
[   11.366727] dracut-pre-pivot[2156]: modprobe: FATAL: Module sha256 not found.
[   11.694913] dracut: FATAL: FIPS integrity test failed
[   11.695485] dracut: Refusing to continue
[   11.605383] dracut-pre-pivot[2156]: Warning: /boot/.vmlinuz-3.10.0-957.el7.x86_64.hmac does not exist
[   11.775222] System halted.

Obviously the hmac file exists in boot, but does not seem available to the crash environment.  It seems we had something similar in bz 909402 


I can test this on RHEL8, but it might take some time to do.

>> Does adding  "-a fips" in dracut_args work?
see /etc/kdump.conf: dracut_args
No, still fails with the same error.

My configuration:

# kdump-config --show
path /var/crash
core_collector makedumpfile -l --message-level 1 -d 31
ext4 UUID=e726e912-a045-4028-b9fd-efdb6da76fc6
dracut_args -a fips

# cat /etc/fstab | grep -i boot
UUID=cad41ff6-0954-4e2b-8f45-91880f94590c /boot                   xfs     defaults        0 0

# df -h | grep -w test
/dev/sda                      2.0G  6.1M  1.8G   1% /test


# lsinitrd /boot/initramfs-$(uname -r)kdump.img /etc/fstab 
/dev/disk/by-uuid/e726e912-a045-4028-b9fd-efdb6da76fc6 /kdumproot//test ext4 defaults 0 2

XFS module is not inserted in kdump initramfs ( for /boot)

# lsinitrd /boot/initramfs-$(uname -r)kdump.img | grep -i xfs | wc -l
0 

# lsinitrd /boot/initramfs-$(uname -r)kdump.img | grep -i fips
Arguments: --hostonly --hostonly-cmdline --hostonly-i18n --hostonly-mode 'strict' -o 'plymouth dash resume ifcfg' --mount '/dev/disk/by-uuid/e726e912-a045-4028-b9fd-efdb6da76fc6 /kdumproot//test ext4 defaults' -a 'fips' --no-hostonly-default-device -f
fips
-rw-r--r--   1 root     root          441 Mar  5 06:37 etc/fipsmodules
-rw-r--r--   1 root     root         1907 Mar  5 06:37 etc/modprobe.d/fips.conf
-rw-r--r--   1 root     root            0 Sep 27 09:47 etc/system-fips
-rwxr-xr-x   1 root     root        15736 Feb 21  2017 usr/bin/fipscheck
drwxr-xr-x   2 root     root            0 Mar  5 06:37 usr/lib64/fipscheck
-rw-r--r--   1 root     root           65 Feb 21  2017 usr/lib64/fipscheck/fipscheck.hmac
-rw-r--r--   1 root     root           65 Feb 21  2017 usr/lib64/fipscheck/libfipscheck.so.1.2.1.hmac
lrwxrwxrwx   1 root     root           26 Mar  5 06:37 usr/lib64/fipscheck/libfipscheck.so.1.hmac -> libfipscheck.so.1.2.1.hmac
-rwxr-xr-x   1 root     root        11344 Feb 21  2017 usr/lib64/libfipscheck.so.1.2.1
lrwxrwxrwx   1 root     root           21 Mar  5 06:37 usr/lib64/libfipscheck.so.1 -> libfipscheck.so.1.2.1
-rwxr-xr-x   1 root     root          354 Sep 12  2013 usr/lib/dracut/hooks/pre-pivot/01-fips-noboot.sh
-rwxr-xr-x   1 root     root          375 Sep 12  2013 usr/lib/dracut/hooks/pre-trigger/01-fips-boot.sh
-rwxr-xr-x   1 root     root         4615 Sep 27 09:46 usr/sbin/fips.sh


After crashing:

# echo c > /proc/sysrq-trigger 

[ 2207.776394] RIP  [<ffffffffb7c62276>] sysrq_handle_crash+0x16/0x20
[ 2207.778615]  RSP <ffff92a8b9babe58>
[ 2207.780401] CR2: 0000000000000000
[    0.534364] do_IRQ: 0.98 No irq handler for vector (irq -1)
[    0.703056] mce: Unable to init device /dev/mcelog (rc: -5)
[    1.661599] dracut-pre-trigger[86]: mount: unknown filesystem type 'xfs'
[    3.039196] systemd-fsck[1905]: /dev/sda: recovering journal
[    3.059937] systemd-fsck[1905]: /dev/sda: clean, 13/131072 files, 26158/524288 blocks
[    3.184669] dracut-pre-pivot[1932]: mkdir: cannot create directory '/boot': File exists
[    3.194212] dracut-pre-pivot[1932]: mount: unknown filesystem type 'xfs' <============ 
[    3.427619] dracut-pre-pivot[1932]: modprobe: FATAL: Module sha1 not found.
[    3.624749] dracut-pre-pivot[1932]: modprobe: FATAL: Module sha256 not found.
[    4.082923] dracut: FATAL: FIPS integrity test failed
[    3.940700] dracut-pre-pivot[    4.085520] dracut: Refusing to continue
[1932]: Warning: /boot/.vmlinuz-3.10.0-957.5.1.el7.x86_64.hmac does not exist
[    4.146427] System halted.

We might need to fix the issue in dracut package.
Not sure if the patch should be for dracut-fips?

We should detect fips and add fips dracut module in kexec-tools, and according to your test , also need a fix in dracut for the missing kernel module.  I noticed below:
Warning: /boot/.vmlinuz-3.10.0-957.5.1.el7.x86_64.hmac

Probably fips depends on this?

Kairui, can you take a look at this?

Thanks!

Seems the dracut fips module is added, but mouting /boot/ failed because of lacking the fs driver, shouldn't the driver be included automatically.

Lukas, any idea about this?

To be honest this question made me a bit uncertain, but if I am not mistaken (and man bootup says the same thing), the kernel is responsible for loading initramdisk. That means that in the normal boot kernel has to read /boot and so it can't have the driver for that partition compiled as a module.

Hmm, boot loader should be able to read /boot and load kernel and initramdisk, then kernel just boot up and jump into init root fs.

Reading the docs below
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/security_guide/chap-federal_standards_and_regulations

User add fips=1 and boot= in kernel cmdline to enable fips, and dracut mount /boot according to boot= provided infomation.

It just works in normal boot maybe because of /boot and root use same file system, and we have root= in cmdline and root is mandatory so dracut just packs the fs module in initrd.

But for kdump we do not need root filesystem in case ssh/nfs dump, thus the module is not packed in.  If this is true probably dracut can add the related fs module in 01fips dracut module because we have to mount /boot in 01fips.

Moving this to dracut, I've submitted a pr to fix it:
https://github.com/dracutdevs/dracut/pull/553/commits/8b6b3efab39a0ccbe918c92a208b86c06680f7f0

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2289