Bug 1430105 - [preview][prod]container do not get original remote client IP address, X-Forwarded-For is incorrect
Summary: [preview][prod]container do not get original remote client IP address, X-Forw...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Online
Classification: Red Hat
Component: Routing
Version: 3.x
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Stefanie Forrester
QA Contact: zhaliu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-03-07 21:26 UTC by Alex Syrnikov
Modified: 2017-05-10 16:54 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-05-10 16:54:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Alex Syrnikov 2017-03-07 21:26:34 UTC
Description of problem: 

I created docker container with http service, but in HTTP request my application did not get original remote client address in X-Forwarded-For header field. Actually container do not get remote address in any field.

Version-Release number of selected component (if applicable):

current developer preview for OpenShift v3 (doker based containers)

How reproducible:

1. create docker image with http service, which will print HTTP request headers on stdout,
2. store that image to hub.docker.com
3. create project and select "deploy image", chuse "Image Name".
4. finish creating project, create route
5. Make HTTP request to service and look for container\'s logs

Actual results:

My request headers
GET / HTTP/1.1
Host: openshift-masterspline-hello-openshift5.44fs.preview.openshiftapps.com
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,ru;q=0.6
X-Forwarded-Host: openshift-masterspline-hello-openshift5.44fs.preview.openshiftapps.com
X-Forwarded-Port: 80
X-Forwarded-Proto: http
Forwarded: for=172.31.0.91;host=openshift-masterspline-hello-openshift5.44fs.preview.openshiftapps.com;proto=http
X-Forwarded-For: 172.31.0.91

Expected results:

The same headers, but with my real IP in X-Forwarded-For
X-Forwarded-For: 84.17.12.94

Comment 1 Jonathan Yu 2017-03-21 21:18:13 UTC
Hi Alex,

Thanks for the report!

It looks like the ELB needs to be configured with the PROXY protocol so that the original IP address is passed through. It seems that HAProxy is setting correct headers, but is unaware that there's a reverse proxy in front, or does not trust the inbound X-Forwarded-For headers due to configuration.

We will take a look and get back to you.

Comment 2 Stefanie Forrester 2017-03-22 14:03:43 UTC
Enabled proxy protocol on router:
oc env dc/router ROUTER_USE_PROXY_PROTOCOL=true

Verified ELBs are using TCP protocol:


$  aws elb describe-load-balancers --load-balancer-name "$lb" |      jq '[.LoadBalancerDescriptions[]|.ListenerDescriptions]'
[
  [
    {
      "Listener": {
        "InstancePort": 80,
        "LoadBalancerPort": 80,
        "Protocol": "TCP",
        "InstanceProtocol": "TCP"
      },
      "PolicyNames": []
    },
    {
      "Listener": {
        "InstancePort": 5000,
        "LoadBalancerPort": 5000,
        "Protocol": "TCP",
        "InstanceProtocol": "TCP"
      },
      "PolicyNames": []
    },
    {
      "Listener": {
        "InstancePort": 443,
        "LoadBalancerPort": 443,
        "Protocol": "TCP",
        "InstanceProtocol": "TCP"
      },
      "PolicyNames": []
    }
  ]
]

Comment 3 zhaliu 2017-03-23 08:06:49 UTC
The route can not be accessed now!

#oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/caddy-docker.json

#oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/edge/service_unsecure.json

#oc expose svc service-unsecure

#oc get route
NAME               HOST/PORT                                               PATH      SERVICES           PORT      TERMINATION
service-unsecure   service-unsecure-alpha.44fs.preview.openshiftapps.com             service-unsecure   http 

#curl http://service-unsecure-alpha.44fs.preview.openshiftapps.com
curl: (52) Empty reply from server

Comment 4 Ben Bennett 2017-03-24 17:47:42 UTC
I think we need to follow this guide to set up the ELB:
  http://docs.aws.amazon.com/elasticloadbalancing/latest/classic/enable-proxy-protocol.html#enable-proxy-protocol-cli

Comment 5 Stefanie Forrester 2017-05-10 16:54:33 UTC
This issue has been fixed in the latest version of OpenShift Online, which is available here:

https://manage.openshift.com/


Note You need to log in before you can comment on or make changes to this bug.