Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2101818 - Create /EFI/redhat symlinks in TFTP directory
Summary: Create /EFI/redhat symlinks in TFTP directory
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Installation
Version: 6.11.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: Satellite QE Team
Marie Hornickova
URL:
Whiteboard:
: 2103104 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-06-28 12:54 UTC by Marek Hulan
Modified: 2024-08-03 04:25 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
Provisioning of `Grub 2 UEFI` host fails with Capsule Servers running on RHEL 8.5 and later versions Due to changes in the `grub2-efi` package, the base directory where `Grub 2` searches for configuration files has changed. Consequently, attempts to provision a `Grub 2 UEFI` host from Capsule Servers running on RHEL 8.5 or later versions of RHEL 8 fail. To work around the issue, create relative symlinks in the `/var/lib/tftpboot` directory as follows: * /var/lib/tftpboot/EFI/redhat -> ../../grub2
Clone Of:
: 2103104 (view as bug list)
Environment:
Last Closed: 2024-04-04 13:54:42 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 35101 0 Urgent New Create /EFI/redhat symlinks in TFTP directory 2022-09-09 13:22:11 UTC
Red Hat Issue Tracker SAT-14186 0 None None None 2022-12-02 13:00:39 UTC

Description Marek Hulan 2022-06-28 12:54:08 UTC
A change in grub2 build config (package grub2-efi file grubx64.efi) changed the base directory where grub2 tries to find configuration files. It used to be the current directory, now it changed to /EFI/redhat (or /EFI/centos or /EFI/fedora etc). While this works fine for local booting, it breaks our PXE/TFTP provisioning.

We could start building our grub via `grub2-mknetdir` which has options to configure where grub should load configuration files from, but then the bootloader will not be signed and SecureBoot will not be possible.

Therefore I suggest a workaround, let's create a *relative symlinks* in the TFTP directory (/var/lib/tftpboot) as follows:

* /var/lib/tftpboot/EFI/redhat -> ../../grub2
* /var/lib/tftpboot/EFI/centos -> ../../grub2
* /var/lib/tftpboot/EFI/fedora -> ../../grub2

Unfortunately every distribution or RH clone has their own ESP EFI subdirectory (which is needed for multi-boot systems), therefore we need to create symlinks for all linux distributions we support.

This was reported multiple times, for example:

https://community.theforeman.org/t/provisioning-bare-metal-host-over-uefi-client-requesting-grub-cfg-in-wrong-location/29053
Alternative solution would be start building grub2 via the mknetdir and dropping support for SecureBoot for all network booting completely. This is what we do for Debian anyway because the grub2 there is not signed anyway, so we would start doing the same for Red Hat systems.

Triage: This is high-impact regression, all Red Hat / clones systems are affected.

Comment 1 Marek Hulan 2022-06-28 12:54:12 UTC
Created from redmine issue https://projects.theforeman.org/issues/35101

Comment 2 Marek Hulan 2022-06-28 12:54:13 UTC
Upstream bug assigned to None

Comment 17 Brad Buckingham 2022-09-09 13:25:27 UTC
*** Bug 2103104 has been marked as a duplicate of this bug. ***

Comment 19 Ewoud Kohl van Wijngaarden 2023-03-24 17:57:47 UTC
Today I spent some time diving into EFI and the grub 2 packaging for an unrelated reason.

The grub sources are patched in RPM sources to have:

    EFIDIR=$(grep ^ID= /etc/os-release | sed -e 's/^ID=//' -e 's/rhel/redhat/' -e 's/\"//g')

Then use /EFI/${EFIDIR}. In this path we can find the resulting grubx64.efi as well.

Looking at the Foreman source, we have https://github.com/theforeman/foreman/blob/develop/app/views/unattended/provisioning_templates/snippet/pxegrub2_mac.erb as a template. It's quite short, so pasting here for completeness:

echo "Trying /httpboot/grub2/grub.cfg-$net_default_mac"
configfile "/httpboot/grub2/grub.cfg-$net_default_mac"

echo "Trying /grub2/grub.cfg-$net_default_mac"
configfile "/grub2/grub.cfg-$net_default_mac"

# The following four statements breaks grub2 and it's no
# longer able to load any file from the base URL.
# Comment them out to be able to use (UEFI/iPXE) HTTP Boot:
# https://bugzilla.redhat.com/show_bug.cgi?id=1763216
echo "Trying grub2/grub.cfg-$net_default_mac"
configfile "grub2/grub.cfg-$net_default_mac"

echo "Trying grub.cfg-$net_default_mac"
configfile "grub.cfg-$net_default_mac"

I see the bug for the relative config file is resolved. Then looking at the tcpdumps from the upstream community (VERY helpful) we see that it's requesting:

/EFI/centos/grub.cfg-01-a4-bb-6d-81-73-3f

Which is the net_default_mac. Knowing that normally it it's located in /EFI/centos and it's a relative filename, I suspect this is what's being triggered.

From grepping through the grub source I get the impression this isn't generated by grub itself, since grub.cfg- doesn't show up in the source at all.

The big question is why it's not requesting the absolute filenames.

It would be very helpful to see the generated files in the tftp root.

To the workaround: on the DHCP server we instruct to use the grubx64.efi file that we also prepare by copying it form the host in Puppet code:

    $grub_efi_path = $facts['os']['name'] ? {
      /Fedora|CentOS|AlmaLinux|Rocky/ => downcase($facts['os']['name']),
      default         => 'redhat',
    }
    $efi_lib_dir = "/boot/efi/EFI/${grub_efi_path}"

This means we know the path can can create a symlink reliably (otherwise grubx64.efi couldn't have been copied). It should also be sufficient to create a single symlink.

So I really wonder what's in the generated grub files in the tftp root and if removing the relative config files would also work/be a solution.

Comment 20 Ewoud Kohl van Wijngaarden 2023-08-22 09:48:37 UTC
Recently I had a setup where I needed to use this myself in production. I did see it is requesting those files, but it wasn't needed for me. However, for me it was the only thing I saw when I didn't build the default PXE configuration file. That was misleading.

Comment 21 Ewoud Kohl van Wijngaarden 2024-04-04 13:54:42 UTC
I haven't been able to come up with a reproducer. I think a common problem is that people don't build the default PXE config after deploying a Capsule, which is the only reproducer I found.

Comment 22 Red Hat Bugzilla 2024-08-03 04:25:03 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.