Bug 1801407

Summary: Router in v4v6 mode puts brackets around IPv4 addresses in the Forwarded header
Product: OpenShift Container Platform Reporter: Miciah Dashiel Butler Masters <mmasters>
Component: NetworkingAssignee: Miciah Dashiel Butler Masters <mmasters>
Networking sub component: router QA Contact: Arvind iyengar <aiyengar>
Status: CLOSED ERRATA Docs Contact:
Severity: low    
Priority: low CC: aiyengar, amcdermo, aos-bugs, bmcelvee, dmace
Version: 4.4   
Target Milestone: ---   
Target Release: 4.8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: ipv6
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-07-27 22:32:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Miciah Dashiel Butler Masters 2020-02-10 19:57:34 UTC
Description of problem:

In v4v6 mode, the router puts brackets around an IPv4 address in the Forwarded header.  RFC 7239 dictates that the header should have either an IPv4 address (without brackets) or an IPv6 address with brackets.

Steps to Reproduce:

1. Disable the ingress operator:

   oc -n openshift-cluster-version scale deployments/cluster-version-operator --replicas=0
   oc -n openshift-ingress-operator scale deploy/ingress-operator --replicas=0

2. Put the router in v4v6 mode:

   oc -n openshift-ingress set env deploy/router-default ROUTER_IP_V4_V6_MODE=v4v6 -c router

3. Deploy an application that echos its headers:

   oc adm new-project test
   oc -n test create -f ~/src/github.com/openshift/origin/test/extended/testdata/router/router-http-echo-server.yaml

4. Send a request to the application:

   host="x.$(oc get ingresses.config/cluster -o 'jsonpath={.spec.domain}')"
   addr="$(dig +short "$host")"
   curl "http://$host" -H 'Host: router-headers.example.com'

Actual results:

The curl output shows an IPv4 address inside brackets:

    GET / HTTP/1.1
    User-Agent: curl/7.29.0
    Accept: */*
    Host: router-headers.example.com
    X-Forwarded-Host: router-headers.example.com
    X-Forwarded-Port: 80
    X-Forwarded-Proto: http
    Forwarded: for="[174.19.21.82]";host=router-headers.example.com;proto=http;proto-version=""
    X-Forwarded-For: 174.19.21.82


Expected results:

The forwarded header should not have brackets around an IPv4 address:

    Forwarded: for=174.19.21.82;host=router-headers.example.com;proto=http;proto-version=""

RFC 7239 permits double-quotes around the value, so the following is also acceptable:

    Forwarded: for="174.19.21.82";host=router-headers.example.com;proto=http;proto-version=""

Comment 1 Miciah Dashiel Butler Masters 2020-05-08 20:10:47 UTC
A proposed fix is posted at https://github.com/openshift/router/pull/83, waiting to be merged.

Comment 4 Miciah Dashiel Butler Masters 2020-06-18 19:24:35 UTC
The proposed fix mentioned in comment 1 is still waiting to be merged, blocked on getting access to an IPv6 cluster for testing.

Comment 5 Miciah Dashiel Butler Masters 2020-07-09 05:12:28 UTC
The proposed fix is still blocked on testing.

Comment 6 Miciah Dashiel Butler Masters 2020-07-30 08:32:03 UTC
The proposed fix is still blocked on testing.

Comment 8 Miciah Dashiel Butler Masters 2020-08-21 05:02:48 UTC
We'll try to get the proposed fix tested this upcoming sprint.

Comment 9 Andrew McDermott 2020-09-10 11:57:23 UTC
I’m adding UpcomingSprint, because I was occupied by fixing bugs with
higher priority/severity, developing new features with higher
priority, or developing new features to improve stability at a macro
level. I will revisit this bug next sprint.

Comment 10 Miciah Dashiel Butler Masters 2020-10-02 14:29:45 UTC
We'll try to get the proposed fix merged in the upcoming sprint.

Comment 11 Miciah Dashiel Butler Masters 2020-10-26 05:31:05 UTC
We'll continue tracking this issue in the upcoming sprint.

Comment 12 Miciah Dashiel Butler Masters 2020-11-14 00:25:05 UTC
Need to investigate CI failures.  I'll investigate in the upcoming sprint.

Comment 13 Miciah Dashiel Butler Masters 2020-12-07 03:03:39 UTC
I'll investigate the CI failures in the upcoming sprint.

Comment 14 Miciah Dashiel Butler Masters 2021-02-06 00:05:44 UTC
Testing revealed a small issue in how IPv4 source addresses are reported on dualstack clusters.  The propose fix has been updated, and we'll continue working on this in the upcoming sprint.

Comment 16 Miciah Dashiel Butler Masters 2021-02-16 05:53:19 UTC
Setting target release to 4.8.0 because this didn't make 4.7.

Comment 17 Arvind iyengar 2021-03-24 10:18:09 UTC
Verified in "4.8.0-0.nightly-2021-03-22-011000" release version. With this payload, the 'Forwarded' field now displays the ipv4 address correctly without any additional brackets: 
----
$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.0-0.nightly-2021-03-22-011000   True        False         38h     Cluster version is 4.8.0-0.nightly-2021-03-22-011000


$ oc get all
NAME                       READY   STATUS      RESTARTS   AGE
pod/header-test-1-9vw86    1/1     Running     0          116s
pod/header-test-1-deploy   0/1     Completed   0          119s

NAME                                  DESIRED   CURRENT   READY   AGE
replicationcontroller/header-test-1   1         1         1       119s

NAME                           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/header-test-insecure   ClusterIP   172.30.181.233   <none>        8080/TCP   37s

NAME                                             REVISION   DESIRED   CURRENT   TRIGGERED BY
deploymentconfig.apps.openshift.io/header-test   1          1         1         config


$ oc expose  service/header-test-insecure
route.route.openshift.io/header-test-insecure exposed

NAME                                            HOST/PORT                                                             PATH   SERVICES               PORT   TERMINATION   WILDCARD
route.route.openshift.io/header-test-insecure   header-test-insecure-test.apps.ocp-edge-cluster-0.qe.lab.redhat.com          header-test-insecure   http                 None


$ curl header-test-insecure-test.apps.ocp-edge-cluster-0.qe.lab.redhat.com
<pre>
  user-agent: curl/7.61.1
  accept: */*
  host: header-test-insecure-test.apps.ocp-edge-cluster-0.qe.lab.redhat.com
  x-forwarded-host: header-test-insecure-test.apps.ocp-edge-cluster-0.qe.lab.redhat.com
  x-forwarded-port: 80
  x-forwarded-proto: http
  forwarded: for=192.168.123.1;host=header-test-insecure-test.apps.ocp-edge-cluster-0.qe.lab.redhat.com;proto=http
  x-forwarded-for: 192.168.123.1
----

Comment 19 Brandi Munilla 2021-06-24 16:51:02 UTC
Hi, does this bug require doc text? If so, please update the doc text field.

Comment 22 errata-xmlrpc 2021-07-27 22:32:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438