Bug 1922565 - EFI HTTP boot fails if the HTTP headers are lower case
Summary: EFI HTTP boot fails if the HTTP headers are lower case
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: shim
Version: 36
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Robbie Harwood
QA Contact: Fedora Extras Quality Assurance
URL: https://github.com/rhboot/shim/pull/403
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-01-30 07:30 UTC by Juan Orti
Modified: 2022-06-17 01:19 UTC (History)
11 users (show)

Fixed In Version: shim-15.6-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-06-17 01:19:02 UTC
Type: Bug


Attachments (Terms of Use)

Description Juan Orti 2021-01-30 07:30:02 UTC
Description of problem:
After adding a reverse proxy (HAproxy) to my EFI HTTP boot setup, my EFI VMs no longer boot with the error:

~~~
>>Start HTTP Boot over IPv4....
  Station IP address is 192.168.5.141

  URI: http://bootserver.lan/grub/shim.efi
  File Size: 1210776 Bytes
  Downloading...100%Failed to get Content-Lenght
Invalid image
Failed to read header: Unsupported
Failed to load image: Unsupported
start_imagee() returned Unsupported
~~~

I've seen that the problem is that HAproxy now returns the content-lenght header in lower case. After adding these options to HAproxy, Content-Lenght is returned capitalized and the boot is working again:

~~~
global
    h1-case-adjust content-length Content-Length

defaults
    option h1-case-adjust-bogus-client
~~~

https://cbonte.github.io/haproxy-dconv/2.2/configuration.html#option%20h1-case-adjust-bogus-client


Version-Release number of selected component (if applicable):
edk2-ovmf-20200801stable-3.fc33.noarch

How reproducible:
Always

Steps to Reproduce:
1. Setup a HTTP boot environment. For example with dnsmasq:

dhcp-vendorclass=set:efi-x86_64-http,HTTPClient:Arch:00016
dhcp-option-force=tag:efi-x86_64-http,option:vendor-class,HTTPClient
dhcp-boot=tag:efi-x86_64-http,"http://bootserver.lan/grub/shim.efi"

2. Have the HTTP server behind HAproxy, which by default returns all headers in lower case

Actual results:
Fail to boot with the error mentioned in the description.

Expected results:
The EFI firmware should process the headers in a case insensitive way as stated by RFC7230.

Additional info:

Comment 1 Laszlo Ersek 2021-02-02 10:47:37 UTC
(Thanks for the CC, Dan.)


Juan, I believe there must be more to this than just a "content-length" / "Content-Length" case difference.

The "Content-Length" header is defined as follows, in edk2's "MdePkg/Include/IndustryStandard/Http11.h":

#define HTTP_HEADER_CONTENT_LENGTH     "Content-Length"

By grepping for HTTP_HEADER_CONTENT_LENGTH, we find several uses; all uses are arguments to the HttpFindHeader() function.

Quoting (part of) that function, from "NetworkPkg/Library/DxeHttpLib/DxeHttpLib.c":

  for (Index = 0; Index < HeaderCount; Index++){
    //
    // Field names are case-insensitive (RFC 2616).
    //
    if (AsciiStriCmp (Headers[Index].FieldName, FieldName) == 0) {
      return &Headers[Index];
    }
  }
  return NULL;

So the header lookup already happens case-insensitively.

I don't know if the edk2 HTTP implementation intends to support HAproxy (or any similar reverse proxy setup). I suggest filing an upstream bug report at <MdePkg/Include/IndustryStandard/Http11.h>, and CC'ing the NetworkPkg maintainer (Maciej Rabeda <maciej.rabeda.com>).

FWIW, I have never even tested the HTTP server of dnsmasq (even without HAproxy). The RHEL[78] manuals describe httpd (Apache); I've tested those (not with HAproxy though). This could be totally uncharted territory (hence my suggestion to file an upstream bug).

Technically, I could of course file the upstream bug myself, but (a) I have no reproducer, (b) more details / configs about the environment will almost certainly be requested by the maintainer; it's best if those come directly from the reporter.

Comment 2 Laszlo Ersek 2021-02-02 10:49:19 UTC
(In reply to Laszlo Ersek from comment #1)

> I suggest filing an upstream bug report at
> <MdePkg/Include/IndustryStandard/Http11.h>, and CC'ing the NetworkPkg
> maintainer (Maciej Rabeda <maciej.rabeda.com>).

Clipboard user malfunction; sorry about that. The upstream bug tracker URL is <https://bugzilla.tianocore.org/>.

Comment 3 Daniel Berrangé 2021-02-02 11:13:18 UTC
(In reply to Juan Orti from comment #0)
> Description of problem:
> After adding a reverse proxy (HAproxy) to my EFI HTTP boot setup, my EFI VMs
> no longer boot with the error:
> 
> ~~~
> >>Start HTTP Boot over IPv4....
>   Station IP address is 192.168.5.141
> 
>   URI: http://bootserver.lan/grub/shim.efi
>   File Size: 1210776 Bytes
>   Downloading...100%Failed to get Content-Lenght
> Invalid image
> Failed to read header: Unsupported
> Failed to load image: Unsupported
> start_imagee() returned Unsupported
> ~~~

Looking at these messages a second time, the "Downloading....100%" part makes me think that ED2 has succesfully downloaded the shim.efi file, and the error messages are coming from the next  phase

Indeed if i look at the shim source https://github.com/rhboot/shim/  then we can find:

  Invalid image
  Failed to read header: Unsupported
  Failed to load image: Unsupported
  start_imagee() returned Unsupported

are all messages in the shim.c source file. 

The one I can't find is the "Failed to get Content-Lenght" message - possibly that comes from UEFI services that shim calls into ?

Comment 4 Juan Orti 2021-02-02 11:19:59 UTC
(In reply to Laszlo Ersek from comment #1)
> Juan, I believe there must be more to this than just a "content-length" /
> "Content-Length" case difference.

Hi Laszlo,

Well, I can reproduce it quite easily with the "content-length" header. Here are all the HTTP headers:

This works:

$ curl -4 -v -o /dev/null http://bootserver.lan/grub/shim.efi
* Connected to bootserver.lan (192.168.5.1) port 80 (#0)
> GET /grub/shim.efi HTTP/1.1
> Host: bootserver.lan
> User-Agent: curl/7.71.1
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: nginx/1.19.6
< date: Tue, 02 Feb 2021 11:00:10 GMT
< content-type: application/efi
< Content-Length: 1210776
< last-modified: Wed, 27 Jan 2021 20:50:22 GMT
< etag: "6011d20e-127998"
< accept-ranges: bytes
< 


When I remove the mentioned workaround in HAproxy, content-length is in lower case and the VM no longer boots with the error of the bug descritpion:

$ curl -4 -v -o /dev/null http://bootserver.lan/grub/shim.efi
* Connected to bootserver.lan (192.168.5.1) port 80 (#0)
> GET /grub/shim.efi HTTP/1.1
> Host: bootserver.lan
> User-Agent: curl/7.71.1
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: nginx/1.19.6
< date: Tue, 02 Feb 2021 11:01:55 GMT
< content-type: application/efi
< content-length: 1210776
< last-modified: Wed, 27 Jan 2021 20:50:22 GMT
< etag: "6011d20e-127998"
< accept-ranges: bytes
< 


> I don't know if the edk2 HTTP implementation intends to support HAproxy (or
> any similar reverse proxy setup). I suggest filing an upstream bug report at
> <MdePkg/Include/IndustryStandard/Http11.h>, and CC'ing the NetworkPkg
> maintainer (Maciej Rabeda <maciej.rabeda.com>).
> 
> FWIW, I have never even tested the HTTP server of dnsmasq (even without
> HAproxy). The RHEL[78] manuals describe httpd (Apache); I've tested those
> (not with HAproxy though). This could be totally uncharted territory (hence
> my suggestion to file an upstream bug).

My HTTP server is nginx with HAproxy in front as reverse proxy. The client has no knowledge if there's a reverse proxy, so I think it shouldn't matter.

> 
> Technically, I could of course file the upstream bug myself, but (a) I have
> no reproducer, (b) more details / configs about the environment will almost
> certainly be requested by the maintainer; it's best if those come directly
> from the reporter.

Sure, I'll open a bug upstream.
Thanks.

Comment 5 Laszlo Ersek 2021-02-02 11:30:17 UTC
(In reply to Daniel Berrangé from comment #3)
> (In reply to Juan Orti from comment #0)
> > Description of problem:
> > After adding a reverse proxy (HAproxy) to my EFI HTTP boot setup, my EFI VMs
> > no longer boot with the error:
> > 
> > ~~~
> > >>Start HTTP Boot over IPv4....
> >   Station IP address is 192.168.5.141
> > 
> >   URI: http://bootserver.lan/grub/shim.efi
> >   File Size: 1210776 Bytes
> >   Downloading...100%Failed to get Content-Lenght
> > Invalid image
> > Failed to read header: Unsupported
> > Failed to load image: Unsupported
> > start_imagee() returned Unsupported
> > ~~~
> 
> Looking at these messages a second time, the "Downloading....100%" part
> makes me think that ED2 has succesfully downloaded the shim.efi file, and
> the error messages are coming from the next  phase

Oops, thanks for catching this; I missed it.

> Indeed if i look at the shim source https://github.com/rhboot/shim/  then we
> can find:
> 
>   Invalid image
>   Failed to read header: Unsupported
>   Failed to load image: Unsupported
>   start_imagee() returned Unsupported
> 
> are all messages in the shim.c source file. 
> 
> The one I can't find is the "Failed to get Content-Lenght" message -
> possibly that comes from UEFI services that shim calls into ?

Both the string in question and the underlying bug are in shim; from commit 3d79bcb2651b ("Add the optional HTTPBoot support", 2016-09-06).

The headers are scanned with the "strcmpa()" function [src/httpboot.c]:

        /* Check the length of the file */
        for (i = 0; i < rx_message.HeaderCount; i++) {
                if (!strcmpa(rx_message.Headers[i].FieldName, (CHAR8 *)"Content-Length")) {
                        *buf_size = ascii_to_int(rx_message.Headers[i].FieldValue);
                }
        }

        if (*buf_size == 0) {
                perror(L"Failed to get Content-Lenght\n");
                goto error;
        }


where "strcmpa()" is a gnu-efi function [lib/str.c]:

UINTN
strcmpa (
    IN CONST CHAR8    *s1,
    IN CONST CHAR8    *s2
    )
// compare strings
{
    while (*s1) {
        if (*s1 != *s2) {
            break;
        }

        s1 += 1;
        s2 += 1;
    }

    return *s1 - *s2;
}

So strcmpa() is case-sensitive, and therefore it is not suitable for the header lookup in shim.

This BZ should be reassigned to the "shim" component.

Comment 6 Ben Cotton 2021-11-04 13:44:47 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 7 Ben Cotton 2021-11-04 14:14:16 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 8 Juan Orti 2021-11-04 14:53:35 UTC
The fix for this bug is merged upstream and I see it has been included in shim 15.5rc1 so hopefully it'll be fixed in a future release.

Comment 9 Ben Cotton 2021-11-04 15:11:53 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 10 Ben Cotton 2022-02-08 21:34:36 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 36 development cycle.
Changing version to 36.

Comment 11 Fedora Update System 2022-06-15 17:42:36 UTC
FEDORA-2022-98830efc68 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2022-98830efc68

Comment 12 Fedora Update System 2022-06-16 02:14:52 UTC
FEDORA-2022-98830efc68 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-98830efc68`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-98830efc68

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 13 Fedora Update System 2022-06-17 01:19:02 UTC
FEDORA-2022-98830efc68 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.