Bug 1852728

Summary: Default 503 error page not conforming to RFCs 2616 and 7230 - CRLF
Product: OpenShift Container Platform Reporter: Simon Krenger <skrenger>
Component: NetworkingAssignee: Andrew McDermott <amcdermo>
Networking sub component: router QA Contact: Arvind iyengar <aiyengar>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: unspecified CC: aiyengar, amcdermo, aos-bugs, bbennett, erich, fkrohn, hongli, sgreene, skrenger, xtian
Version: 3.11.0   
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: HAProxy router 503 page contained LF line endings instead of CRLF endings. Consequence: Some Web Application Firewalls, etc. are unable to consume the 503 page from the router since it technically is not spec compliant. Fix: Change the line endings of the HAProxy router 503 page (whitespace only change) Result: HAProxy 503 page is now spec compliant.
 Story Points: ---
Clone Of:
: 1881133 (view as bug list) Environment:
Last Closed: 2020-10-27 16:10:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1879120, 1881133    

Description Simon Krenger 2020-07-01 07:50:49 UTC 
Description of problem:

The default 503 error page in Openshift's HAproxy template contains linebreaks in LF format, whereas the RFCs 2616 and 7230 both demand for CRLF linebreaks to be used:

RFC 2616 - https://tools.ietf.org/html/rfc2616#section-19.3 ("(19.3) The line terminator for message-header fields is the sequence CRLF.")
RFC 7230 - https://tools.ietf.org/html/rfc7230#section-3

openshift's 503 contains LF instead of CRLF in the image as well as in the files published on github from hich the images are generated:

    $ wget https://raw.githubusercontent.com/openshift/router/master/images/router/haproxy/conf/error-page-503.http -O- | xxd | head -n3
    00000000: 4854 5450 2f31 2e30 2035 3033 2053 6572  HTTP/1.0 503 Ser
    00000010: 7669 6365 2055 6e61 7661 696c 6162 6c65  vice Unavailable
    00000020: 0a50 7261 676d 613a 206e 6f2d 6361 6368  .Pragma: no-cach

(Note: CR is 0d in hex, LF is 0a, CRLF is 0d0a)

upstream HAproxy also contains a default 503 error page, which is RFC-compliant:

    $ wget -q "http://git.haproxy.org/?p=haproxy-1.8.git;a=blob_plain;f=examples/errorfiles/503.http;hb=HEAD" -O- | xxd | head -n3
    00000000: 4854 5450 2f31 2e30 2035 3033 2053 6572  HTTP/1.0 503 Ser
    00000010: 7669 6365 2055 6e61 7661 696c 6162 6c65  vice Unavailable
    00000020: 0d0a 4361 6368 652d 436f 6e74 726f 6c3a  ..Cache-Control:

While those RFCs also stipulate that applications MAY be tolerant and accept only CR or LF linebreaks, this does not help for proxies and WAFs who check on 100% compliance and may fail otherwise.

The customer has already submitted a pull request upstream to fix this issue: https://github.com/openshift/router/pull/140

Version-Release number of selected component (if applicable):

OpenShift Container Platform 3.11.216

How reproducible:

Always

Steps to Reproduce:
1. Execute "wget https://raw.githubusercontent.com/openshift/router/master/images/router/haproxy/conf/error-page-503.http -O- | xxd | head -n3" or request any page from the OpenShift Container Platform Cluster that will produce a HTTP 503 error.

Actual results:

Observe only a "CR" is sent, not a "CRLF":

    $ wget https://raw.githubusercontent.com/openshift/router/master/images/router/haproxy/conf/error-page-503.http -O- | xxd | head -n3
    00000000: 4854 5450 2f31 2e30 2035 3033 2053 6572  HTTP/1.0 503 Ser
    00000010: 7669 6365 2055 6e61 7661 696c 6162 6c65  vice Unavailable
    00000020: 0a50 7261 676d 613a 206e 6f2d 6361 6368  .Pragma: no-cach


Expected results:

upstream HAproxy also contains a default 503 error page, which is RFC-compliant:

    $ wget -q "http://git.haproxy.org/?p=haproxy-1.8.git;a=blob_plain;f=examples/errorfiles/503.http;hb=HEAD" -O- | xxd | head -n3
    00000000: 4854 5450 2f31 2e30 2035 3033 2053 6572  HTTP/1.0 503 Ser
    00000010: 7669 6365 2055 6e61 7661 696c 6162 6c65  vice Unavailable
    00000020: 0d0a 4361 6368 652d 436f 6e74 726f 6c3a  ..Cache-Control:


Additional info:

There is already a Pull Request opened upstream to fix this issue: https://github.com/openshift/router/pull/140
Comment 3 Andrew McDermott 2020-07-09 12:12:16 UTC 
I’m adding UpcomingSprint, because I was occupied by fixing bugs with
higher priority/severity, developing new features with higher
priority, or developing new features to improve stability at a macro
level. I will revisit this bug next sprint.
Comment 4 Andrew McDermott 2020-07-30 10:07:40 UTC 
I’m adding UpcomingSprint, because I was occupied by fixing bugs with
higher priority/severity, developing new features with higher
priority, or developing new features to improve stability at a macro
level. I will revisit this bug next sprint.
Comment 5 mfisher 2020-08-18 19:55:53 UTC 
Target reset to 4.7 while investigation is either ongoing or not yet started.  Will be considered for earlier release versions when diagnosed and resolved.
Comment 6 Stephen Greene 2020-08-18 20:02:12 UTC 
this is related to https://issues.redhat.com/browse/RFE-140
Comment 13 Arvind iyengar 2020-09-15 09:57:08 UTC 
This bug has been verified in "4.6.0-0.nightly-2020-09-12-230035" release. With this version, it is noted that the headers are now contains CR-LF linebreaks:
-----
$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.0-0.nightly-2020-09-12-230035   True        False         27h     Cluster version is 4.6.0-0.nightly-2020-09-12-230035

sh-4.4$ haproxy -v 
HA-Proxy version 2.0.16 2020/07/17 - https://haproxy.org/

sh-4.4$ hexdump -C /var/lib/haproxy/conf/error-page-503.http  | head -n3
00000000  48 54 54 50 2f 31 2e 30  20 35 30 33 20 53 65 72  |HTTP/1.0 503 Ser|
00000010  76 69 63 65 20 55 6e 61  76 61 69 6c 61 62 6c 65  |vice Unavailable|
00000020  0d 0a 50 72 61 67 6d 61  3a 20 6e 6f 2d 63 61 63  |..Pragma: no-cac|
------
Comment 15 errata-xmlrpc 2020-10-27 16:10:31 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196