Description of problem: The 'oc rsh' and 'oc exec' commands make two http requests. While both go through the proxy (specified by the http_proxy & https_proxy environment variables) only the first of the two includes a Proxy-Authorization header with the necessary proxy credentials. How reproducible: Always. Steps to Reproduce: 1. Be behind an authenticated proxy server 2. Set proper proxy environment variables: export http_proxy='http://user:pass@proxy:port' export https_proxy='http://user:pass@proxy:port' 3. Run 'oc rsh $POD' Actual results: Will fail with an error like: "error: error sending request: Post https://ose3-master.devday.osecloud.com:8443/api/v1/namespaces/mycliproject-imuser13/pods/time-1-py9ub/exec?command=%2Fbin%2Fbash&container=time&container=time&stderr=true&stdin=true&stdout=true&tty=true: EOF" Expected results: Open an ssh session on the pod. Additional info: The HTTP headers from the first connection to the proxy are: CONNECT ose3-master.devday.osecloud.com:8443 HTTP/1.1 Host: ose3-master.devday.osecloud.com:8443 User-Agent: Go 1.1 package http Proxy-Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ= The HTTP headers from the second connection to the proxy are: CONNECT ose3-master.devday.osecloud.com:8443 HTTP/1.1 Host: ose3-master.devday.osecloud.com:8443 User-Agent: Go 1.1 package http The only difference is that the second request lacked Proxy-Authorization and so the proxy server returns an HTTP 407 response: HTTP/1.1 407 Proxy Authentication Required Server: squid/3.3.13 Mime-Version: 1.0 Date: Thu, 28 Jan 2016 21:57:04 GMT Content-Type: text/html Content-Length: 340 X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 Vary: Accept-Language Content-Language: en Proxy-Authenticate: NTLM Proxy-Authenticate: Basic realm="Intermountain Health Care Proxy (Note: Password is case sensitive)" X-Cache: MISS from proxy.ihc.com Via: 1.1 proxy.ihc.com (squid/3.3.13) Connection: close
Related to https://bugzilla.redhat.com/show_bug.cgi?id=1302422
Created attachment 1119496 [details] localhost proxy to add missing Proxy-Authorization header Usage: Modify lines 8-10 of bug-1302894-workaround.tcl with your authenticated proxy information, then run ./bug-1302894-workaround.tcl In another Window: export set http_proxy=http://localhost:8080 export set https_proxy=http://localhost:8080 Then run: oc rsh $POD This will take the credentials provided from oc's first CONNECT request and include them with the second allowing the 'oc rsh' (or 'oc exec') command to work.
Comment on attachment 1119496 [details] localhost proxy to add missing Proxy-Authorization header Usage: Edit lines 8-10 of bug-1302894-workaround.tcl to specify your authenticated proxy Run ./bug-1302894-workaround.tcl In another window, run: export set http_proxy=http://user:pass@localhost:8080 export set https_proxy=http://user:pass@localhost:8080 oc rsh $POD
Created attachment 1119500 [details] --v=9 level logging with workaround in place
Created attachment 1119501 [details] --v=9 level logging WITHOUT workaround
Fixed in https://github.com/openshift/origin/pull/7362.
Verified against latest Origin, the bug is fixed. The latest version of OSE has not yet merged the fix in. Will verify against OSE when it merged.
Verified against latest OSE, versions are: oc v3.1.1.905 kubernetes v1.2.0-alpha.7-703-gbc4550d Verification steps: 1. (If there is no existing one) prepare a proxy server that needs authentication 1> $ sudo yum install squid 2> $ sudo htpasswd -c /etc/squid/passwd xxia 3> $ sudo vi /etc/squid/squid.conf # Add the following lines in proper places: auth_param basic program /usr/lib64/squid/basic_ncsa_auth /etc/squid/passwd acl SSL_ports port 8443 acl squid_user proxy_auth xxia http_access allow squid_user 4> $ sudo service squid restart 2. oc login and create project 3. Create pod (from new app) $ oc new-app -f origin/examples/sample-app/application-template-stibuild.json 4. Check `oc rsh`, `oc exec` behind authenticated proxy 1> $ export http_proxy=xxia:<password>@<proxy server>:3128 $ export https_proxy=xxia:<password>@<proxy server>:3128 2> $ oc rsh database-1-uqnvn bash-4.2$ $ oc exec database-1-uqnvn ls /etc/hosts /etc/hosts Actual results: 4.2 Both `oc rsh`, `oc exec` commands succeed. The bug is fixed, so move it to VERIFIED.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2016:1064