Bug 1665015

Summary: [Lenovo 8.1 FEAT] OS Network launching via HTTP Boot
Product: Red Hat Enterprise Linux 8 Reporter: Kelvin Shieh <kshieh>
Component: anacondaAssignee: Anaconda Maintenance Team <anaconda-maint-list>
Status: CLOSED NOTABUG QA Contact: Release Test Team <release-test-team>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.1CC: achen35, bootloader-eng-team, cchang13, dhsia, fmartine, jkachuck, jstodola, kshieh, mknutson, mtsai2, tyu1
Target Milestone: alphaKeywords: FutureFeature, Reopened
Target Release: 8.1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-04-09 07:49:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1490991, 1649271, 1681759    
Bug Blocks: 1613899, 1615852    
Attachments:
Description Flags
rhel 8.1_Pre-Beta_http boot log1
none
rhel 8.1_Pre-Beta_http boot log2
none
sosreport
none
grub boot
none
install failed
none
serial log none

Description Kelvin Shieh 2019-01-10 09:37:45 UTC 
Description of problem:
Network launching via HTTP Boot failed on RHEL8 beta, we would need to make sure RHEL8.1 will support this feature


Version-Release number of selected component (if applicable):
4.18.0-32.el8.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Set up a HTTP Boot server
2. Use the core.efi or grubx64.efi as the NBP.
3. Validate the HTTP Boot function on a server with HTTP Boot function supported.

Actual results:
Boot failed and enter grub rescue mode.

Expected results:
Boot successfully via HTTP boot.

Additional info:
Comment 1 Javier Martinez Canillas 2019-06-13 11:56:23 UTC 
This seems to be a duplicate of Bug #1490991.

*** This bug has been marked as a duplicate of bug 1490991 ***
Comment 2 Michelle Tsai 2019-06-17 09:22:11 UTC 
Created attachment 1581373 [details]
rhel 8.1_Pre-Beta_http boot log1
Comment 3 Michelle Tsai 2019-06-17 09:22:33 UTC 
Created attachment 1581374 [details]
rhel 8.1_Pre-Beta_http boot log2
Comment 4 Michelle Tsai 2019-06-17 09:24:37 UTC 
Hi Javier,

The issue is still occur when we used RHEL 8.1 Pre-Beta (InternalSnapshot-3) with grub2 version 73 (grub2-efi-x64-cdboot-2.02-73.el8.x86_64.rpm).
Please take a look at the logs, see Comment 2 & 3.

Thanks.
Comment 5 Javier Martinez Canillas 2019-06-17 12:04:20 UTC 
Hello Michelle,

(In reply to Michelle Tsai from comment #4)
> Hi Javier,
> 
> The issue is still occur when we used RHEL 8.1 Pre-Beta (InternalSnapshot-3)
> with grub2 version 73 (grub2-efi-x64-cdboot-2.02-73.el8.x86_64.rpm).
> Please take a look at the logs, see Comment 2 & 3.
>

Thanks a lot for the logs. I see that the error is slightly different now though.

In the rhel8.1_Alpha_log.PNG file attached in Bug 1649271, Comment 15 the error message was:

error: ../../grub-core/net/efi/http.c:234:Fail to send a request! status=0x9

So GRUB was failing to send an HTTP request using the EFI firmware.


But in your logs from Comment 2 and Comment 3, the error message is:

error: ../../grub-core/net/efi/http.c:277:Fail to receive a response! status=0x9

In this case GRUB isn't able to receive response of the HTTP request.

Could you please provide the access and error logs of the HTTP server?

I also see in your attached logs error messages like the following:

error: ../../grub-core/disk/efi/efidisk.c:599:failure reading sector 0x8b771ff0 from `hd3`.
Comment 7 Michelle Tsai 2019-06-18 10:04:26 UTC 
Created attachment 1581622 [details]
sosreport
Comment 8 Michelle Tsai 2019-06-18 10:06:03 UTC 
(In reply to Michelle Tsai from comment #7)
> Created attachment 1581622 [details]
> sosreport

Hi Javier,
Please have the sosreport log of HTTP server. Thanks!
Comment 9 Javier Martinez Canillas 2019-06-18 10:17:12 UTC 
Hello Michelle,

Thanks a lot for the logs. In the httpd access.log I see the following:

192.168.100.206 - - [17/Jun/2019:05:01:58 -0400] "HEAD /rhel8.1/EFI/BOOT/BOOTX64.EFI HTTP/1.1" 200 - "-" "UefiHttpBoot/1.0"
192.168.100.206 - - [17/Jun/2019:05:01:58 -0400] "GET /rhel8.1/EFI/BOOT/BOOTX64.EFI HTTP/1.1" 200 1211224 "-" "UefiHttpBoot/1.0"
192.168.100.206 - - [17/Jun/2019:05:02:03 -0400] "GET /rhel8.1/EFI/BOOT//grubx64.efi HTTP/1.1" 200 1734072 "-" "UefiHttpBoot/1.0"
192.168.100.206 - - [17/Jun/2019:05:02:03 -0400] "HEAD /rhel8.1/EFI/BOOT/grub.cfg HTTP/1.1" 200 - "-" "UefiHttpBoot/1.0"
192.168.100.206 - - [17/Jun/2019:05:02:03 -0400] "GET /rhel8.1/EFI/BOOT/grub.cfg HTTP/1.1" 200 1412 "-" "UefiHttpBoot/1.0"
192.168.100.206 - - [17/Jun/2019:05:02:03 -0400] "HEAD /EFI/BOOT/x86_64-efi/command.lst HTTP/1.1" 404 - "-" "UefiHttpBoot/1.0"
192.168.100.206 - - [17/Jun/2019:05:02:03 -0400] "HEAD /EFI/BOOT/x86_64-efi/fs.lst HTTP/1.1" 404 - "-" "UefiHttpBoot/1.0"
192.168.100.206 - - [17/Jun/2019:05:02:03 -0400] "HEAD /EFI/BOOT/x86_64-efi/crypto.lst HTTP/1.1" 404 - "-" "UefiHttpBoot/1.0"
192.168.100.206 - - [17/Jun/2019:05:02:03 -0400] "HEAD /EFI/BOOT/x86_64-efi/terminal.lst HTTP/1.1" 404 - "-" "UefiHttpBoot/1.0"
192.168.100.206 - - [17/Jun/2019:05:02:03 -0400] "HEAD /rhel8.1/EFI/BOOT/grub.cfg HTTP/1.1" 200 - "-" "UefiHttpBoot/1.0"
192.168.100.206 - - [17/Jun/2019:05:02:03 -0400] "GET /rhel8.1/EFI/BOOT/grub.cfg HTTP/1.1" 200 1412 "-" "UefiHttpBoot/1.0"
192.168.100.206 - - [17/Jun/2019:05:02:09 -0400] "HEAD /images/pxeboot/vmlinuz HTTP/1.1" 404 - "-" "UefiHttpBoot/1.0"

So it seems shim (/rhel8.1/EFI/BOOT/BOOTX64.EFI), grub (/rhel8.1/EFI/BOOT//grubx64.efi) and the grub config (/rhel8.1/EFI/BOOT/grub.cfg) is fetched correctly. But the kernel image (/images/pxeboot/vmlinuz HTTP/1.1) isn't found. Do you have that file in /var/www/html ?
Comment 10 Michelle Tsai 2019-06-19 10:07:06 UTC 
Hi Javier,

We copy the "vmlinuz" file from directory "/var/www/html/rhel8.1/images/pxeboot/" into "/var/www/html/images/pxeboot/".
Then re-connect the HTTP server from another Client, the result is failed as well. But there are something different results in the access_logs & Client, you might check it:

<HTTP Server>

"HEAD /images/pxeboot/vmlinuz HTTP/1.1" 404 - "-" "UefiHttpBoot/1.0"

change to

"HEAD /images/pxeboot/vmlinuz HTTP/1.1" 200 - "-" "UefiHttpBoot/1.0"

192.168.100.208 - - [18/Jun/2019:06:32:17 -0400] "HEAD /rhel8.1/EFI/BOOT/BOOTX64.EFI HTTP/1.1" 200 - "-" "UefiHttpBoot/1.0"
192.168.100.208 - - [18/Jun/2019:06:32:17 -0400] "GET /rhel8.1/EFI/BOOT/BOOTX64.EFI HTTP/1.1" 200 1211224 "-" "UefiHttpBoot/1.0"
192.168.100.208 - - [18/Jun/2019:06:32:18 -0400] "GET /rhel8.1/EFI/BOOT//grubx64.efi HTTP/1.1" 200 1734072 "-" "UefiHttpBoot/1.0"
192.168.100.208 - - [18/Jun/2019:06:32:18 -0400] "HEAD /rhel8.1/EFI/BOOT/grub.cfg HTTP/1.1" 200 - "-" "UefiHttpBoot/1.0"
192.168.100.208 - - [18/Jun/2019:06:32:18 -0400] "GET /rhel8.1/EFI/BOOT/grub.cfg HTTP/1.1" 200 1412 "-" "UefiHttpBoot/1.0"
192.168.100.208 - - [18/Jun/2019:06:32:18 -0400] "HEAD /EFI/BOOT/x86_64-efi/command.lst HTTP/1.1" 404 - "-" "UefiHttpBoot/1.0"
192.168.100.208 - - [18/Jun/2019:06:32:18 -0400] "HEAD /EFI/BOOT/x86_64-efi/fs.lst HTTP/1.1" 404 - "-" "UefiHttpBoot/1.0"
192.168.100.208 - - [18/Jun/2019:06:32:18 -0400] "HEAD /EFI/BOOT/x86_64-efi/crypto.lst HTTP/1.1" 404 - "-" "UefiHttpBoot/1.0"
192.168.100.208 - - [18/Jun/2019:06:32:18 -0400] "HEAD /EFI/BOOT/x86_64-efi/terminal.lst HTTP/1.1" 404 - "-" "UefiHttpBoot/1.0"
192.168.100.208 - - [18/Jun/2019:06:32:18 -0400] "HEAD /rhel8.1/EFI/BOOT/grub.cfg HTTP/1.1" 200 - "-" "UefiHttpBoot/1.0"
192.168.100.208 - - [18/Jun/2019:06:32:18 -0400] "GET /rhel8.1/EFI/BOOT/grub.cfg HTTP/1.1" 200 1412 "-" "UefiHttpBoot/1.0"
192.168.100.208 - - [18/Jun/2019:06:32:22 -0400] "HEAD /images/pxeboot/vmlinuz HTTP/1.1" 200 - "-" "UefiHttpBoot/1.0"

------------------------------------------------------------------------------------------------------------------------------------

<Client>

"error: ../../grub-core/net/efi/http.c:277:Fail to receive a response! status=0x9"

change to

error: ../../grub-core/net/efi/http.c:234:Fail to send a request! status=0x9

------------------------------------------------------------------------------------------------------------------------------------
Comment 11 Javier Martinez Canillas 2019-06-19 10:58:00 UTC 
Hello Michelle,

Could you please attach a screenshot of the GRUB boot log in the case when the HTTP request for vmlinuz succeeds?
Comment 12 Michelle Tsai 2019-06-20 03:52:55 UTC 
Created attachment 1582515 [details]
grub boot
Comment 13 Michelle Tsai 2019-06-20 03:53:17 UTC 
Created attachment 1582516 [details]
install failed
Comment 14 Michelle Tsai 2019-06-20 04:04:58 UTC 
Hi Javier,

We can into GRUB boot (see attachment 1582515 [details]) but fail to install RHEL 8.1 with following error message(see attachment 1582516 [details]) .

error: ../../grub-core/net/efi/http.c:234:Fail to send a request! status=0xf

error: ../../grub-core/loader/i386/efi/linux.c:208:/images/pxeboot/vmlinuz has invalid sigature.
error: ../../grub-core/loader/i386/efi/linux.c:93:you need to load the kernel first.

Press any key to continue...


Not sure why it has invalid signature in here. Can you please help to clarify it?

Thanks!
Comment 15 Michelle Tsai 2019-07-30 08:15:28 UTC 
It looks like the symptom also occurs on latest RHEL 8.1 Beta 1.0. Thanks.



(In reply to Michelle Tsai from comment #14)
> Hi Javier,
> 
> We can into GRUB boot (see attachment 1582515 [details]) but fail to install
> RHEL 8.1 with following error message(see attachment 1582516 [details]) .
> 
> error: ../../grub-core/net/efi/http.c:234:Fail to send a request! status=0xf
> 
> error: ../../grub-core/loader/i386/efi/linux.c:208:/images/pxeboot/vmlinuz
> has invalid sigature.
> error: ../../grub-core/loader/i386/efi/linux.c:93:you need to load the
> kernel first.
> 
> Press any key to continue...
> 
> 
> Not sure why it has invalid signature in here. Can you please help to
> clarify it?
> 
> Thanks!
Comment 16 Michelle Tsai 2019-07-31 02:47:04 UTC 
Hi, do we have plan to support "HTTP Boot" feature on RHEL 8.1? or move to RHEL 8.2? Thank you.
Comment 17 Javier Martinez Canillas 2019-11-21 14:43:14 UTC 
Hello Michelle,

(In reply to Michelle Tsai from comment #16)
> Hi, do we have plan to support "HTTP Boot" feature on RHEL 8.1? or move to
> RHEL 8.2? Thank you.

The HTTP Boot feature itself is supported and works with most EFI machines. But it's still not clear if the issue you are facing is in GRUB or the EFI firmware.
Comment 18 Michelle Tsai 2019-11-26 08:29:56 UTC 
Hi Javier,

Right now we can boot into GRUB menu and install the 8.1 by HTTP boot but failed on Anaconda when loading install.img & squashfs.img, please see the log for more details...
Thanks.


[   78.639251] dracut-initqueue[1767]: Warning: can't find installer main image path in .treeinfo

[   78.657366] dracut-initqueue[1767]:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

[   78.657489] dracut-initqueue[1767]:                                  Dload  Upload   Total   Spent    Left  Speed

[   78.659161] dracut-initqueue[1767]: 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 1 seconds. 3 retries left.

[   79.660944] dracut-initqueue[1767]: 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 2 seconds. 2 retries left.

[   81.663436] dracut-initqueue[1767]: 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 4 seconds. 1 retries left.

[   85.668106] dracut-initqueue[1767]: 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: www.httpboot.local

[   85.673044] dracut-initqueue[1767]: Warning: Downloading 'http://www.httpboot.local/rhel8.1/images/install.img' failed!

[   85.692584] dracut-initqueue[1767]:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

[   85.695350] dracut-initqueue[1767]:                                  Dload  Upload   Total   Spent    Left  Speed

[   85.695624] dracut-initqueue[1767]: 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 1 seconds. 3 retries left.

[   86.696466] dracut-initqueue[1767]: 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 2 seconds. 2 retries left.

[   88.698937] dracut-initqueue[1767]: 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 4 seconds. 1 retries left.

[   92.703543] dracut-initqueue[1767]: 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: www.httpboot.local

[   92.708512] dracut-initqueue[1767]: Warning: Downloading 'http://www.httpboot.local/rhel8.1/LiveOS/squashfs.img' failed!

[   92.711062] dracut-initqueue[1767]: Warning: anaconda: failed to fetch stage2 from http://www.httpboot.local/rhel8.1
Comment 19 Michelle Tsai 2019-11-26 08:31:39 UTC 
Created attachment 1639723 [details]
serial log
Comment 20 Javier Martinez Canillas 2019-12-02 13:59:20 UTC 
Hello Michelle,

It seems the problem is that www.httpboot.local can't be resolved. From your logs:


[   85.668106] dracut-initqueue[1767]: 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: www.httpboot.local


But in any case this isn't a GRUB bug anymore so I'll move the component to anaconda.
Comment 21 Michelle Tsai 2019-12-06 08:14:46 UTC 
(In reply to Javier Martinez Canillas from comment #20)
> Hello Michelle,
> 
> It seems the problem is that www.httpboot.local can't be resolved. From your
> logs:
> 
> 
> [   85.668106] dracut-initqueue[1767]: 
>   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--    
> 0curl: (6) Could not resolve host: www.httpboot.local
> 
> 
> But in any case this isn't a GRUB bug anymore so I'll move the component to
> anaconda.

Got it, thanks Javier! Please also let me know if there is any further update on this.
Comment 22 Jan Stodola 2020-02-12 19:04:29 UTC 
Are you dropped into dracut shell after the installer fails to access www.httpboot.local? What is the network configuration in dracut shell?
# ip a
# ip r
# cat /etc/resolv.conf

Are you able to download .treeinfo using curl?
# curl http://www.httpboot.local/rhel8.1/.treeinfo
Comment 23 Jan Stodola 2020-04-02 12:42:02 UTC 
Michelle, please see comment 22 - could you provide additional info in case you are still able to reproduce it?
Comment 24 Michelle Tsai 2020-04-03 00:46:59 UTC 
(In reply to Jan Stodola from comment #23)
> Michelle, please see comment 22 - could you provide additional info in case
> you are still able to reproduce it?

Thank you for your reminder & sorry for late response.
Yes, I will verify it next week since we're in holiday from 4/2 to 4/5.
Comment 25 Michelle Tsai 2020-04-03 00:56:43 UTC 
By the way, I notice that Bug 1490991 in Comment 69 (as below), do I need to validation HTTP boot with new grub2-efi package? Thank you.


Javier Martinez Canillas 2020-03-23 12:09:37 UTC
Created attachment 1672665 [details]
grub2-efi-x64-cdboot-2.02-82.el8.x86_64.rpm

Can you please give a try to the attached grub2-efi-x64-cdboot-2.02-82.el8.x86_64.rpm ? This contains several fixes for the HTTP boot support.
Comment 26 Jan Stodola 2020-04-03 13:19:33 UTC 
Michelle, please try it with grub2-efi-x64-cdboot present in RHEL-8.2 (grub2-efi-x64-cdboot-2.02-81.el8).
Based on your comment 18, it looks like a problem with network configuration.
In addition to information requested in comment 22, please provide also:

# cat /proc/cmdline
Comment 27 Michelle Tsai 2020-04-09 06:22:21 UTC 
@ Jan, after verified the issue with RHEL 8.2 Snapshot3, now we are able to install 8.2 with HTTP Boot.
Please advise if there is anything need to check from my site.
Thank you.
Comment 28 Jan Stodola 2020-04-09 06:54:37 UTC 
Thanks for retesting and confirming it works for you.
It seems all problems reported in this bug have been clarified and resolved and this bug can be closed, am I correct?
Comment 29 Michelle Tsai 2020-04-09 07:20:40 UTC 
(In reply to Jan Stodola from comment #28)
> Thanks for retesting and confirming it works for you.
> It seems all problems reported in this bug have been clarified and resolved
> and this bug can be closed, am I correct?

Yes, it works and this bug can be closed now. Thank you.


Michelle
Comment 30 Jan Stodola 2020-04-09 07:49:58 UTC 
Closing this bug, the feature is working, there were configuration issues.