Bug 2015146

Summary: Leapp can fail when /boot is a multipath
Product: Red Hat Enterprise Linux 7 Reporter: Christophe Besson <cbesson>
Component: leapp-repositoryAssignee: Leapp Notifications Bot <leapp-notifications-bot>
Status: NEW --- QA Contact: upgrades-and-conversions
Severity: medium Docs Contact:
Priority: medium    
Version: 7.9CC: rmetrich, smitterl
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Christophe Besson 2021-10-18 14:00:37 UTC
Description of problem:
During the reboot step, Leapp is unable to mount /boot, it uses a wrong device name. After that, it is unable to proceed with the first actor (remove_grub_boot_entry).

Version-Release number of selected component (if applicable):
leapp-repository-0.14.0-4.el7_9.noarch

How reproducible:
Always for the customer.

Actual results:
~~~~~~~~~
[...]
Sep 28 03:57:49 localhost upgrade[3457]: mount: /dev/sdk2 is already mounted or /boot busy
Sep 28 03:57:49 localhost kernel: XFS (dm-17): Mounting V5 Filesystem
Sep 28 03:57:49 localhost upgrade[3457]: mount: mount point /boot/efi does not exist
Sep 28 03:57:49 localhost kernel: XFS (dm-17): Ending clean mount
Sep 28 03:57:49 localhost kernel: XFS (dm-18): Mounting V5 Filesystem
Sep 28 03:57:49 localhost kernel: XFS (dm-18): Ending clean mount
Sep 28 03:57:49 localhost kernel: XFS (dm-15): Mounting V5 Filesystem
Sep 28 03:57:49 localhost kernel: XFS (dm-15): Ending clean mount
Sep 28 03:57:49 localhost kernel: XFS (dm-16): Mounting V5 Filesystem
Sep 28 03:57:49 localhost kernel: XFS (dm-16): Ending clean mount
Sep 28 03:57:54 localhost upgrade[3457]: ==> Processing phase `InitRamStart`
Sep 28 03:57:54 localhost upgrade[3457]: ====> * remove_upgrade_boot_entry
Sep 28 03:57:54 localhost upgrade[3457]:         Remove boot entry for Leapp provided initramfs.
Sep 28 03:57:54 localhost upgrade[3457]: Process Process-165:
Sep 28 03:57:54 localhost upgrade[3457]: Traceback (most recent call last):
[...]
Sep 28 03:57:54 localhost upgrade[3457]:   File "/usr/lib/python2.7/site-packages/leapp/libraries/stdlib/__init__.py", line 188, in run
Sep 28 03:57:54 localhost upgrade[3457]:     result=result
Sep 28 03:57:54 localhost upgrade[3457]: CalledProcessError: Command ['/usr/sbin/grubby', '--remove-kernel=/boot/vmlinuz-upgrade.x86_64'] failed with exit code 1.
Sep 28 03:57:55 localhost upgrade[3457]: ==========================================================================================================
Sep 28 03:57:55 localhost upgrade[3457]: Actor remove_upgrade_boot_entry unexpectedly terminated with exit code: 1 - Please check the above details
Sep 28 03:57:55 localhost upgrade[3457]: ==========================================================================================================
[...]
~~~~~~~~~~~~~~

Additional info:
* The issue here is the fact /dev/sdk2 does not correspond to the boot partition
~~~
sda                                       8:0    0  100G  0 disk  
`-<WWID>     253:9    0  100G  0 mpath 
  |-<WWID>p1 253:10   0  256M  0 part  /boot/efi
  |-<WWID>p2 253:11   0  500M  0 part  /boot
 :
sdk                                       8:160  0  100G  0 disk  
`-<WWID>     253:9    0  100G  0 mpath 
  |-<WWID>p1 253:10   0  256M  0 part  /boot/efi
  |-<WWID>p2 253:11   0  500M  0 part  /boot
~~~

* From my understanding, it's a timing issue with multipathd. The customer have lots of disks behind an Hitachi storage array, and only /boot and /boot/efi are not part of LVM.
* Changing the fstab to use the /dev/mapper/path (with the wwids in this case) fixed the issue. The fstab entries have been reverted back once the system has been upgraded.

Comment 5 Christophe Besson 2023-02-14 16:34:36 UTC
This bug is blocked by:

    Bug 2169770 - System boot hangs after mounting /sysroot when multipath is enabled in the initramfs
    https://bugzilla.redhat.com/show_bug.cgi?id=2169770