Bug 1886215

Summary: Need to disable HTX when not using HTTP/2 in order to preserve HTTP header name case
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: NetworkingAssignee: Miciah Dashiel Butler Masters <mmasters>
Networking sub component: router QA Contact: Arvind iyengar <aiyengar>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: aiyengar, aos-bugs
Version: 4.4   
Target Milestone: ---   
Target Release: 4.4.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-01-13 18:14:32 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1886214    
Bug Blocks:    

Comment 1 Miciah Dashiel Butler Masters 2020-10-26 05:06:05 UTC
4.4.z backport is posted and blocked on verification of the 4.5.z backport.

Comment 2 Miciah Dashiel Butler Masters 2020-11-14 00:50:00 UTC
The bump to HAProxy 2.2 may block this change if the bump affects OCP 4.4 because HAProxy 2.2 removes the "http-use-htx" option (see bug 1897039).  Need to investigate in the upcoming sprint.

Comment 3 Miciah Dashiel Butler Masters 2020-11-16 22:01:20 UTC
(In reply to Miciah Dashiel Butler Masters from comment #2)
> The bump to HAProxy 2.2 may block this change if the bump affects OCP 4.4
> because HAProxy 2.2 removes the "http-use-htx" option (see bug 1897039). 
> Need to investigate in the upcoming sprint.

The bump to HAProxy 2.2 only affects OpenShift 4.7 because it requires a change to the Dockerfiles, and that change has not been backported to earlier release branches, so we can go ahead with turning off HTX in 4.4.

Comment 4 Miciah Dashiel Butler Masters 2020-12-07 03:13:10 UTC
The 4.5 backport is verified, but the 4.4 backport is waiting for lgtm and cherry-pick-approval labels.  We'll try to get this done in the upcoming sprint.

Comment 5 Arvind iyengar 2020-12-10 09:54:43 UTC
Tested in "4.4.0-0.ci.test-2020-12-10-085627-ci-ln-q8q318t" release. With this payload, it is noted that the haproxy configuration now has the "no option http-use-htx" with http2 disabled, and the headers are no more being moderated for the lower/upper cases:
-----
$ oc get clusterversion
NAME      VERSION                                           AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.4.0-0.ci.test-2020-12-10-085627-ci-ln-q8q318t   True        False         12m     Cluster version is 4.4.0-0.ci.test-2020-12-10-085627-ci-ln-q8q318t

$ oc -n openshift-ingress exec router-internalapps-5b54d6b76b-6tfmw -- cat haproxy.config | grep -i htx                  
  no option http-use-htx

$ oc get all
NAME                            READY   STATUS      RESTARTS   AGE
pod/router-http-echo-1-deploy   0/1     Completed   0          17m
pod/router-http-echo-1-r5r8j    1/1     Running     0          17m

NAME                                       DESIRED   CURRENT   READY   AGE
replicationcontroller/router-http-echo-1   1         1         1       17m

NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/router-http-echo   ClusterIP   172.30.131.121   <none>        8676/TCP   17m

NAME                                                  REVISION   DESIRED   CURRENT   TRIGGERED BY
deploymentconfig.apps.openshift.io/router-http-echo   1          1         1         config

NAME                                        HOST/PORT                                                                                              PATH   SERVICES           PORT               TERMINATION   WILDCARD
route.route.openshift.io/router-http-echo   router-http-echo-test1.internalapps.ci-ln-q8q318t-d5d6b.origin-ci-int-aws.dev.rhcloud.com ... 1 more          router-http-echo   router-http-echo                 None


$ curl router-http-echo-test1.internalapps.ci-ln-q8q318t-d5d6b.origin-ci-int-aws.dev.rhcloud.com -H X-foo:foo -H X-Bar:Bar -H X-BaZ:BaZ
GET / HTTP/1.1
User-Agent: curl/7.68.0
Accept: */*
X-foo:foo
X-Bar:Bar
X-BaZ:BaZ
Host: router-http-echo-test1.internalapps.ci-ln-q8q318t-d5d6b.origin-ci-int-aws.dev.rhcloud.com
X-Forwarded-Host: router-http-echo-test1.internalapps.ci-ln-q8q318t-d5d6b.origin-ci-int-aws.dev.rhcloud.com
X-Forwarded-Port: 80
X-Forwarded-Proto: http
Forwarded: for=182.70.60.193;host=router-http-echo-test1.internalapps.ci-ln-q8q318t-d5d6b.origin-ci-int-aws.dev.rhcloud.com;proto=http
X-Forwarded-For: 182.70.60.193
-----

Comment 9 errata-xmlrpc 2021-01-13 18:14:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.4.32 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:0029