Bug 2290388 - Unable to send network packet when PXE booting Alma 8
Summary: Unable to send network packet when PXE booting Alma 8
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: edk2
Version: 40
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Paolo Bonzini
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2024-06-04 14:35 UTC by Tim Bosse
Modified: 2024-06-21 01:15 UTC (History)
6 users (show)

Fixed In Version: edk2-20240524-3.fc40 edk2-20240524-2.fc39
Clone Of:
Environment:
Last Closed: 2024-06-11 01:50:07 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
libvirt domain xml dump (9.00 KB, text/plain)
2024-06-05 15:36 UTC, Tim Bosse
no flags Details

Description Tim Bosse 2024-06-04 14:35:15 UTC
When PXE booting to alma8, I see "../../grub-core/net/drivers/efi/efinet.c:85:couldn't send network" when I upgraded F40 to edk2-ovmf-20240524-1.fc40.noarch

I originally thought this might be a grub issue, however, downgrading edk2-ovmf to edk2-ovmf-20240214-7.fc40.noarch resolves the issue.  I no longer see the error and I am able to continue network booting. 

Reproducible: Always

Steps to Reproduce:
1. Verify edk2-ovmf-20240524-1.fc40.noarch installed
2. Attempt to IPv4 PXE boot from UEFI.  Grub chainloads, but then returns the error.
Actual Results:  
Grub error:  "../../grub-core/net/drivers/efi/efinet.c:85:couldn't send network"

Expected Results:  
Boot into alma8.

I'm going to follow-up with other tests for different distributions/versions.  It'll have to wait for when I have free time as this bug affects my day job.  Downgrading  the version of edk2-ovmf resolves the issue for me.  I'm not familiar enough with how to help troubleshoot further, but I am willing to walk through this.

Comment 1 Tim Bosse 2024-06-04 14:36:03 UTC
My copy is bad.  The full error is "../../grub-core/net/drivers/efi/efinet.c:85:couldn't send network packet"

Comment 2 Gerd Hoffmann 2024-06-04 14:52:55 UTC
The edk2 network stack got a dependency on a random number source (EFI_RNG_PROTOCOL)
for security reasons (to make outgoing port numbers *really* random for example).

The random number source for OVMF is virtio-rng, so adding a virtio-rng-pci device
to your guest will fix the network boot failure.  libvirt should do that by default.
When running qemu directly simply add '-device virtio-rng-pci' to the command line.

Independent from the above there is upstream discussion / development in progress
to bring support for the rdrand cpu instruction to OVMF, which would allow to get
real random numbers even without virtio-rng on sufficiently recent CPUs.

Comment 3 Tim Bosse 2024-06-04 15:01:15 UTC
I already have the following added to the libvirt domain definition:

    <rng model="virtio">
      <backend model="random">/dev/urandom</backend>
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </rng>

Would that have been sufficient?

Comment 4 Gerd Hoffmann 2024-06-04 15:23:39 UTC
(In reply to Tim Bosse from comment #3)
> I already have the following added to the libvirt domain definition:
> 
>     <rng model="virtio">
>       <backend model="random">/dev/urandom</backend>
>       <address type="pci" domain="0x0000" bus="0x05" slot="0x00"
> function="0x0"/>
>     </rng>
> 
> Would that have been sufficient?

Yes, that should be it.

Reading the initial report again I see loading grub.efi via
network works, so the edk2 network stack seems to be fine.

So apparently something trips up grub.  There is a good
chance that this is somehow related to the RNG change.

Which grub binary is this?  The one from alma 8 I assume?
Where does it fail?  When fetching the grub.cfg file or later?

Comment 5 Tim Bosse 2024-06-04 15:35:32 UTC
From an alma8 guest:

    # rpm -qa "grub2-*"
    grub2-tools-2.02-123.el8_6.8.alma.x86_64
    grub2-tools-extra-2.02-123.el8_6.8.alma.x86_64
    grub2-tools-minimal-2.02-123.el8_6.8.alma.x86_64
    grub2-common-2.02-123.el8_6.8.alma.noarch

I'll get back to you on where it fails later today.  I'll capture boot output, which I should have done in the first place.

Comment 6 Fedora Update System 2024-06-05 12:17:08 UTC
FEDORA-2024-3446df5831 (edk2-20240524-3.fc40) has been submitted as an update to Fedora 40.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-3446df5831

Comment 7 Fedora Update System 2024-06-05 13:30:27 UTC
FEDORA-2024-773ea76c63 (edk2-20240524-2.fc39) has been submitted as an update to Fedora 39.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-773ea76c63

Comment 8 Tim Bosse 2024-06-05 15:33:58 UTC
Sorry, I tried again last night, but I was getting a bit of a different experience.  I get the following:

    BdsDxe: failed to load Boot0001 "UEFI QEMU HARDDISK QM00001 " from PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0xFFFF,0x0): Not Found

    >>Start PXE over IPv4.
      Station IP address is 192.168.124.12

      Server IP address is 192.168.124.1
      NBP filename is machines/a171e644-3896-48e4-9627-ecd37ed559d1/boot/97064f6a618603ffd6f37670dafa81b41a00b5cc/shimx64.efi
      NBP filesize is 944576 Bytes
     Downloading NBP file...

      NBP file downloaded successfully.
    BdsDxe: loading Boot0002 "UEFI PXEv4 (MAC:525400BF9159)" from PciRoot(0x0)/Pci(0x2,0x0)/Pci(0x0,0x0)/MAC(525400BF9159,0x1)/IPv4(0.0.0.0,0x0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0)
    BdsDxe: starting Boot0002 "UEFI PXEv4 (MAC:525400BF9159)" from PciRoot(0x0)/Pci(0x2,0x0)/Pci(0x0,0x0)/MAC(525400BF9159,0x1)/IPv4(0.0.0.0,0x0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0)
    Fetching Netboot Image

Then it boots after many minutes (I walked away after 15 minutes, came back an hour later) and it booted.  Nothing I could capture as the serial console seems to wipe away whatever was there after kernel boot?

I then updated to edk2-20240524-3.fc40, tested again, I'm now getting the same experience of a really long wait on "Fetching Netboot Image".

Last thing I tried this morning was to go back to disable uefi, I get the following now:


    >>Start PXE over IPv4.
      Station IP address is 192.168.124.12

      Server IP address is 192.168.124.1
      NBP filename is machines/a171e644-3896-48e4-9627-ecd37ed559d1/boot/97064f6a618603ffd6f37670dafa81b41a00b5cc/shimx64.efi
      NBP filesize is 944576 Bytes
     Downloading NBP file...

      NBP file downloaded successfully.
    BdsDxe: loading Boot0002 "UEFI PXEv4 (MAC:525400BF9159)" from PciRoot(0x0)/Pci(0x2,0x0)/Pci(0x0,0x0)/MAC(525400BF9159,0x1)/IPv4(0.0.0.0,0x0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0)
    BdsDxe: starting Boot0002 "UEFI PXEv4 (MAC:525400BF9159)" from PciRoot(0x0)/Pci(0x2,0x0)/Pci(0x0,0x0)/MAC(525400BF9159,0x1)/IPv4(0.0.0.0,0x0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0)
    Fetching Netboot Image
    error: ../../grub-core/net/drivers/efi/efinet.c:85:couldn't send network packet.
    error: ../../grub-core/loader/i386/efi/linux.c:232:you need to load the kernel first.
    error: ../../grub-core/commands/boot.c:197:you need to load the kernel first.
          Minimal BASH-like line editing is supported. For the first word, TAB lists possible
          command completions. Anywhere else TAB lists possible device or file completions.


    grub>

Comment 9 Tim Bosse 2024-06-05 15:36:21 UTC
Created attachment 2036428 [details]
libvirt domain xml dump

Hopefully helps describe my current setup.

Comment 10 Tim Bosse 2024-06-05 15:51:33 UTC
I'll try to spend some time tonight getting a tcpdump of the whole thing.

Comment 11 Tim Bosse 2024-06-05 17:22:36 UTC
The patch from comment 6 works for me.  I didn't understand how the packages were setup, and what I needed to do with koji to get the right packages for the most recent update.  I was only grabbing edk2-tools with x86_64.

I downloaded the correct ovmf package using 'koji download-build --arch=noarch edk2-20240524-3.fc40'

I tested everything and all are booting as expected, and in a timely fashion.  I'll throw some karma on the build.  Thanks for patching and watching me stumble through testing.  I've put this package on my rss feed for future karma testing now that I know what to do.

Comment 12 Fedora Update System 2024-06-06 02:12:51 UTC
FEDORA-2024-3446df5831 has been pushed to the Fedora 40 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-3446df5831`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-3446df5831

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 13 Fedora Update System 2024-06-06 03:09:16 UTC
FEDORA-2024-773ea76c63 has been pushed to the Fedora 39 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-773ea76c63`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-773ea76c63

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 14 Fedora Update System 2024-06-11 01:50:07 UTC
FEDORA-2024-3446df5831 (edk2-20240524-3.fc40) has been pushed to the Fedora 40 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 15 Fedora Update System 2024-06-21 01:15:26 UTC
FEDORA-2024-773ea76c63 (edk2-20240524-2.fc39) has been pushed to the Fedora 39 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.