Bug 1302894 - [behind proxy] 'oc rsh' and 'oc exec' fail behind an authenticated proxy
[behind proxy] 'oc rsh' and 'oc exec' fail behind an authenticated proxy
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Command Line Interface (Show other bugs)
3.1.0
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Fabiano Franz
Wei Sun
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-28 17:07 EST by Michael A. Cleverly
Modified: 2017-01-21 22:03 EST (History)
12 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-05-12 12:27:30 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
localhost proxy to add missing Proxy-Authorization header (1.65 KB, application/x-shellscript)
2016-01-29 13:46 EST, Michael A. Cleverly
no flags Details
--v=9 level logging with workaround in place (5.46 KB, text/plain)
2016-01-29 14:13 EST, Michael A. Cleverly
no flags Details
--v=9 level logging WITHOUT workaround (5.31 KB, text/plain)
2016-01-29 14:14 EST, Michael A. Cleverly
no flags Details

  None (edit)
Description Michael A. Cleverly 2016-01-28 17:07:52 EST
Description of problem:

The 'oc rsh' and 'oc exec' commands make two http requests.  While both go through the proxy (specified by the http_proxy & https_proxy environment variables) only the first of the two includes a Proxy-Authorization header with the necessary proxy credentials.

How reproducible:

Always.

Steps to Reproduce:

1. Be behind an authenticated proxy server

2. Set proper proxy environment variables:

    export http_proxy='http://user:pass@proxy:port'
    export https_proxy='http://user:pass@proxy:port'

3. Run 'oc rsh $POD'


Actual results:

Will fail with an error like: "error: error sending request: Post https://ose3-master.devday.osecloud.com:8443/api/v1/namespaces/mycliproject-imuser13/pods/time-1-py9ub/exec?command=%2Fbin%2Fbash&container=time&container=time&stderr=true&stdin=true&stdout=true&tty=true: EOF"

Expected results:

Open an ssh session on the pod.

Additional info:

The HTTP headers from the first connection to the proxy are:

    CONNECT ose3-master.devday.osecloud.com:8443 HTTP/1.1
    Host: ose3-master.devday.osecloud.com:8443
    User-Agent: Go 1.1 package http
    Proxy-Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=

The HTTP headers from the second connection to the proxy are:

    CONNECT ose3-master.devday.osecloud.com:8443 HTTP/1.1
    Host: ose3-master.devday.osecloud.com:8443
    User-Agent: Go 1.1 package http

The only difference is that the second request lacked Proxy-Authorization and so the proxy server returns an HTTP 407 response:

    HTTP/1.1 407 Proxy Authentication Required
    Server: squid/3.3.13
    Mime-Version: 1.0
    Date: Thu, 28 Jan 2016 21:57:04 GMT
    Content-Type: text/html
    Content-Length: 340
    X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
    Vary: Accept-Language
    Content-Language: en
    Proxy-Authenticate: NTLM
    Proxy-Authenticate: Basic realm="Intermountain Health Care Proxy (Note: Password is case sensitive)"
    X-Cache: MISS from proxy.ihc.com
    Via: 1.1 proxy.ihc.com (squid/3.3.13)
    Connection: close
Comment 1 Fabiano Franz 2016-01-29 08:44:38 EST
Related to https://bugzilla.redhat.com/show_bug.cgi?id=1302422
Comment 2 Michael A. Cleverly 2016-01-29 13:46 EST
Created attachment 1119496 [details]
localhost proxy to add missing Proxy-Authorization header

Usage: 

Modify lines 8-10 of bug-1302894-workaround.tcl with your authenticated proxy information, then run ./bug-1302894-workaround.tcl

In another Window:

export set http_proxy=http://localhost:8080
export set https_proxy=http://localhost:8080

Then run: oc rsh $POD 

This will take the credentials provided from oc's first CONNECT request and include them with the second allowing the 'oc rsh' (or 'oc exec') command to work.
Comment 3 Michael A. Cleverly 2016-01-29 14:03:58 EST
Comment on attachment 1119496 [details]
localhost proxy to add missing Proxy-Authorization header

Usage: 

Edit lines 8-10 of bug-1302894-workaround.tcl to specify your authenticated proxy

Run ./bug-1302894-workaround.tcl

In another window, run:

export set http_proxy=http://user:pass@localhost:8080
export set https_proxy=http://user:pass@localhost:8080
oc rsh $POD
Comment 4 Michael A. Cleverly 2016-01-29 14:13 EST
Created attachment 1119500 [details]
--v=9 level logging with workaround in place
Comment 5 Michael A. Cleverly 2016-01-29 14:14 EST
Created attachment 1119501 [details]
--v=9 level logging WITHOUT workaround
Comment 6 Fabiano Franz 2016-02-19 19:12:47 EST
Fixed in https://github.com/openshift/origin/pull/7362.
Comment 7 Xingxing Xia 2016-02-22 04:30:43 EST
Verified against latest Origin, the bug is fixed.
The latest version of OSE has not yet merged the fix in. Will verify against OSE when it merged.
Comment 8 Xingxing Xia 2016-02-22 22:14:22 EST
Verified against latest OSE, versions are:
oc v3.1.1.905
kubernetes v1.2.0-alpha.7-703-gbc4550d

Verification steps:
1. (If there is no existing one) prepare a proxy server that needs authentication
1> $ sudo yum install squid
2> $ sudo htpasswd -c /etc/squid/passwd xxia
3> $ sudo vi /etc/squid/squid.conf # Add the following lines in proper places:
auth_param basic program /usr/lib64/squid/basic_ncsa_auth /etc/squid/passwd

acl SSL_ports port 8443

acl squid_user proxy_auth xxia

http_access allow squid_user
4> $ sudo service squid restart

2. oc login and create project

3. Create pod (from new app)
$ oc new-app -f origin/examples/sample-app/application-template-stibuild.json

4. Check `oc rsh`, `oc exec` behind authenticated proxy
1>
$ export http_proxy=xxia:<password>@<proxy server>:3128
$ export https_proxy=xxia:<password>@<proxy server>:3128
2>
$ oc rsh database-1-uqnvn
bash-4.2$
$ oc exec database-1-uqnvn ls /etc/hosts
/etc/hosts

Actual results:
4.2 Both `oc rsh`, `oc exec` commands succeed.

The bug is fixed, so move it to VERIFIED.
Comment 11 errata-xmlrpc 2016-05-12 12:27:30 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2016:1064

Note You need to log in before you can comment on or make changes to this bug.