Bug 1612340 - Kernel used by upgrade tool will not boot if FIPS is enabled
Summary: Kernel used by upgrade tool will not boot if FIPS is enabled
Status: ASSIGNED
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: preupgrade-assistant-el6toel7
Version: 6.10
Hardware: Unspecified
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: pstodulk
QA Contact: Alois Mahdal
Lenka Špačková
URL:
Whiteboard:
Keywords: Extras
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-08-03 17:15 UTC by Welterlen Benoit
Modified: 2019-04-12 04:20 UTC (History)
9 users (show)

(edit)
In-place upgrade from a RHEL 6 system to RHEL 7.6 is impossible with FIPS mode enabled

When upgrading a RHEL 6 system to RHEL 7.6 using the Red Hat Upgrade Tool with FIPS mode enabled, missing Hash-based Message Authentication Code (HMAC) prevents kernel data from being correctly verified. As a consequence, Red Hat Upgrade Tool cannot boot into the target system kernel and the process fails. To work around this problem, disable FIPS mode by removing "fips=1" from the kernel command line and re-enable it after the upgrade.
Clone Of:
(edit)
Last Closed:


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3552141 Troubleshoot None Kernel used by redhat-upgrade-tool will not boot if FIPS is enabled 2019-04-11 07:55 UTC

Description Welterlen Benoit 2018-08-03 17:15:24 UTC
Description of problem:
The kernel of the upgrader will no boot if FIPS is enabled.

Version-Release number of selected component (if applicable):
RHEL 6.x to RHEL7.x with FIPS enabled

How reproducible:
Easy, always

Steps to Reproduce:
Following the documentation here to upgrade RHEL6.10 to RHEL 7.5:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/migration_planning_guide/chap-red_hat_enterprise_linux-migration_planning_guide-upgrading

1. install or update to Red Hat 6.10, enable FIPS
2. Triggering redhat-upgrade-tool: redhat preupgrade scritps 
3. reboot to the upgrader
The kernel will not boot complaining about the .hmac missing


Actual results:
- not able to boot the upgrader kernel, and so the upgrade is not possible without disabling FIPS

Expected results:
- upgrade without issue

Additional info:
Workaround:
- The workaround is, before the reboot to proceed to the update, to extract the .hmac file from the RPM (from the repo or the ISO) and the kernel also because it is identified as vmlinuz-3.10.0-862.el7.x86_64.

For example if the ISO is mounted on /mnt:
# rpm2cpio /mnt/Packages/kernel-3.10.0-862.el7.x86_64.rpm | cpio -iv --to-stdout ./boot/.vmlinuz-3.10.0-862.el7.x86_64.hmac > /boot/.vmlinuz-3.10.0-862.el7.x86_64.hmac
# rpm2cpio /mnt/Packages/kernel-3.10.0-862.el7.x86_64.rpm | cpio -iv --to-stdout ./boot/vmlinuz-3.10.0-862.el7.x86_64 > /boot/vmlinuz-3.10.0-862.el7.x86_64	

Then the reboot to the kernel of the updater tool will work.

Comment 2 pstodulk 2018-09-24 09:57:36 UTC
I see the issue. Unfortunately our team is in in time pressure so the solution will be postponed probably till the RHEL 7.7 GA time and the Known Issue will be published, meanwhile.

Comment 3 pstodulk 2018-10-17 08:48:42 UTC
Hi Welterlen,
we tried your workround manytimes (even with various additional changes) but we have not been able to proceed the upgrade. Every time we've got warning that:
  /boot/.vmlinuz-3.10.0-957.el7.x86_64.hmac
does not exists. Even when we see that we are able to find it on the mentioned path with the same permissions as the others. The vmlinuz has been put there as well. Only workround that work for us is set fips=0 in kernel commandline.

As we are not able to fix it now and even use your workround, we will document the "fips=0" solution as a possible workround in known issue. Any help here is welcomed.

Comment 4 Welterlen Benoit 2018-10-17 09:01:59 UTC
Hello,

When debugging the issue, I was able to use the workaround.
Did you extract the same kernel that the one used during the boot ? I can see that your version is not the same than mine. 
Are the names and paths correct ?

Thank you

Benoit

Comment 5 pstodulk 2018-10-17 09:19:25 UTC
Yes,
we extracted the files from the same kernel rpm that is used during the upgrade process (in case of upgrade using the network you can find the rpm in /system-upgrade directory). We checked in two people that paths has been correct. I can try to proceed the upgrade again later with clean virtual machine to let you know.

My version-release of kernel is different as we are working on upgrades RHEL 6.10 -> RHEL 7.6 nowadays.


Note You need to log in before you can comment on or make changes to this bug.