Bug 1472976

Summary: [3.6] X-Forwarded-For and related headers send the IPv6 form of the source IPv4 address
Product: OpenShift Container Platform Reporter: Ben Bennett <bbennett>
Component: NetworkingAssignee: Phil Cameron <pcameron>
Networking sub component: router QA Contact: zhaozhanqi <zzhao>
Status: CLOSED ERRATA Docs Contact:
Severity: urgent    
Priority: urgent CC: aloughla, aos-bugs, atragler, eparis, hongli, rkhan, smunilla, sukulkar, zzhao
Version: 3.6.0   
Target Milestone: ---   
Target Release: 3.6.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: haproxy was sending IPv6 addresses in X-Forwarded headers when in ipv6 mode Consequence: The behavior changed so clients that didn't expect ipv6 would break Fix: IPv6 mode has to be enabled manually rather than defaulting to on Result: The headers do not change unexpectedly
Story Points: ---
Clone Of: 1471255
: 1477673 (view as bug list) Environment:
Last Closed: 2017-10-25 13:02:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1471255    
Bug Blocks: 1477673    

Description Ben Bennett 2017-07-19 17:30:35 UTC
+++ This bug was initially created as a clone of Bug #1471255 +++

Description of problem:

The X-Forwarded-For and related headers send the IPv6 form of the source IPv4 address.

e.g.:

	Forwarded: for=::ffff:172.17.0.3;host=hello-openshift-default.router.default.svc.cluster.local;proto=http
	X-Forwarded-For: ::ffff:172.17.0.3



Version-Release number of selected component (if applicable):

3.6.0


How reproducible:

Always


Steps to Reproduce:
1. Set up a router and a route
2. Run tcpdump to sniff the traffic on the target node, and watch for the http request from the proxy
3. curl --ipv4 172.17.0.3 -H 'Host: hello-openshift-default.router.default.svc.cluster.local'


Actual results:

	GET / HTTP/1.1
	User-Agent: curl/7.51.0
	Accept: */*
	Host: hello-openshift-default.router.default.svc.cluster.local
	X-Forwarded-Host: hello-openshift-default.router.default.svc.cluster.local
	X-Forwarded-Port: 80
	X-Forwarded-Proto: http
	Forwarded: for=::ffff:172.17.0.3;host=hello-openshift-default.router.default.svc.cluster.local;proto=http
	X-Forwarded-For: ::ffff:172.17.0.3



Expected results:

	GET / HTTP/1.1
	User-Agent: curl/7.51.0
	Accept: */*
	Host: hello-openshift-default.router.default.svc.cluster.local
	X-Forwarded-Host: hello-openshift-default.router.default.svc.cluster.local
	X-Forwarded-Port: 80
	X-Forwarded-Proto: http
	Forwarded: for=172.17.0.3;host=hello-openshift-default.router.default.svc.cluster.local;proto=http
	X-Forwarded-For: 172.17.0.3




Additional info:

--- Additional comment from Ben Bennett on 2017-07-17 07:25:35 EDT ---

PR https://github.com/openshift/origin/pull/15229

Comment 1 Eric Paris 2017-07-20 16:06:37 UTC
https://github.com/openshift/origin/pull/15351

Comment 3 Hongan Li 2017-07-31 06:41:12 UTC
verified in atomic-openshift-3.6.172.0.1-1.git.0.c989645.el7.x86_64, the test result as below:

1. ROUTER_IP_V4_V6_MODE=v4 (by default)
  forwarded: for=10.8.241.19;host=header-test-insecure-y3f5x.0731-360.qe.rhcloud.com;proto=http
  x-forwarded-for: 10.8.241.19

2. ROUTER_IP_V4_V6_MODE=v4v6
  forwarded: for=::ffff:10.8.241.19;host=header-test-insecure-y3f5x.0731-360.qe.rhcloud.com;proto=http
  x-forwarded-for: ::ffff:10.8.241.19

or

  forwarded: for=2031::31;host=header-test-insecure-y3f5x.0731-360.qe.rhcloud.com;proto=http
  x-forwarded-for: 2031::31

3. ROUTER_IP_V4_V6_MODE=v6
  forwarded: for="[2031::31]";host=header-test-insecure-y3f5x.0731-360.qe.rhcloud.com;proto=http
  x-forwarded-for: 2031::31

From the output we can see if set ROUTER_IP_V4_V6_MODE=v4v6, then the ipv6 address of "forwarded: for=" was not the "[%[src]]" format.


code: 
+  {{- if matchPattern "v6" $router_ip_v4_v6_mode }}
+  # See the quoting rules in https://tools.ietf.org/html/rfc7239 for IPv6 addresses (v4 addresses get translated to v6 when in hybrid mode)
+  http-request set-header Forwarded for="[%[src]]";host=%[req.hdr(host)];proto=%[req.hdr(X-Forwarded-Proto)]
+  {{- else }}

Comment 4 Ben Bennett 2017-07-31 15:50:06 UTC
Ok, so the bug is that 'v4v6' mode does not properly escape.  Given that this is not a regression, I've moved this to 3.6.1.  PR will be posted shortly to correct this behavior.

Comment 5 Ben Bennett 2017-07-31 19:50:19 UTC
@pcameron: Can you please make sure that https://github.com/openshift/origin/pull/14858 lands upstream and then make a backport to 3.6.1 when that cuts to resolve this bug.

Comment 6 Ben Bennett 2017-07-31 19:51:02 UTC
@pcameron: Argh.  I pasted the wrong link.  https://github.com/openshift/origin/pull/15566 is the correct one.

Comment 8 openshift-github-bot 2017-08-07 08:13:07 UTC
Commits pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/e55988d1ecede732aafa054ffde48308995f4030
Corrected the regex used for the v4v6 env parsing

Before the v4v6 case was not correctly matching the regex so the Forwarded header was not correctly escaped for the v6 IP addresses.  This patch corrects the code to parse v4v6 properly.

Fixes bug 1472976 (https://bugzilla.redhat.com/show_bug.cgi?id=1472976)

https://github.com/openshift/origin/commit/586257a30a95d45e1a1d3ac9a783e6fee28eeb17
Merge pull request #15566 from knobunc/bug/bz1472976-escape-ipv6-forward-for

Automatic merge from submit-queue

Corrected the regex used for the v4v6 env parsing

Before the v4v6 case was not correctly matching the regex so the Forwarded header was not correctly escaped for the v6 IP addresses.  This patch corrects the code to parse v4v6 properly.

Fixes bug 1472976 (https://bugzilla.redhat.com/show_bug.cgi?id=1472976)

Comment 10 Hongan Li 2017-09-28 01:10:02 UTC
verified in atomic-openshift-3.6.173.0.37-1.git.0.fd828e7.el7.x86_64 and bug has been fixed.

with "v4v6" mode, the header is:
  forwarded: for="[::ffff:54.172.201.171]";host=header-test-insecure-u1p1.0919-d4z.qe.rhcloud.com;proto=http
  x-forwarded-for: ::ffff:54.172.201.171

or 

  forwarded: for="[3006::36]";host=header-test-insecure-u1p1.0919-d4z.qe.rhcloud.com;proto=http
  x-forwarded-for: 3006::36

Comment 13 errata-xmlrpc 2017-10-25 13:02:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3049