Description of problem:
Version-Release number of the following components:
Steps to Reproduce:
1. Create a VPC
2. Launch a mitm proxy in this VPC
3. Trigger an installation enabled proxy
cloud provider could not be initialized
# oc -n openshift-kube-controller-manager get po
NAME READY STATUS RESTARTS AGE
kube-controller-manager-jialiu43-mitm-11142317-m-0.c.openshift-qe.internal 2/3 CrashLoopBackOff 17 33m
kube-controller-manager-jialiu43-mitm-11142317-m-1.c.openshift-qe.internal 1/3 CrashLoopBackOff 20 32m
kube-controller-manager-jialiu43-mitm-11142317-m-2.c.openshift-qe.internal 2/3 CrashLoopBackOff 21 34m
installation is passed.
This issue is blocking QE's testing against cluster behind mitm proxy
@Daneyon, do we have any update on this issue?
googleapi.com is not automatically added to the default no proxy list. It appears the ca trust bundle being used by controllermanager does not include the ca that signed the proxy's cert. Can you confirm that you created the ca cert configmap and that you referenced the configmap in proxy.spec.trustedCA?
Reproduced this issue using payload 4.3.0-0.nightly-2019-12-04-214544
No configmap with config.openshift.io/inject-trusted-cabundle="true" label created under openshift-kube-controller-manager namespace.
# oc get cm -n openshift-kube-controller-manager -o yaml |grep inject-trusted-cabundle
So this looks like some remaining tasks that need to be completed from kube-controller-manager side for fully proxy support.
Based on the discussion I've had with mfojtik, and what was stated already in https://bugzilla.redhat.com/show_bug.cgi?id=1772756#c2 it's a network bug, and googleapi.com should be whitelisted in NO_PROXY.
Moving this back to networking. KCM should never reach to any external location, except for cloud providers.
The controller manager should add trusted ca support so it can proxy https connections. By doing so, cluster admins will have choice whether or not to proxy gcloud api calls. If not, then add https://www.googleapis.com/compute to the default no_proxy lis. Note that product docs will need to be updated to reflect this requirement. Another option is to require https://cloud.google.com/vpc/docs/private-access-options for gce with mitm proxy.
Working on a fix...
Verify this bug with payload 4.4.0-0.nightly-2019-12-14-103510
# oc get cm trusted-ca-bundle -n openshift-kube-controller-manager -o yaml
# oc get pod -n openshift-kube-controller-manager
NAME READY STATUS RESTARTS AGE
installer-4-qe-gpei-proxy-12160505-m-1.c.openshift-qe.internal 0/1 Completed 0 4h42m
installer-5-qe-gpei-proxy-12160505-m-0.c.openshift-qe.internal 0/1 Completed 0 4h40m
installer-5-qe-gpei-proxy-12160505-m-2.c.openshift-qe.internal 0/1 Completed 0 4h42m
installer-6-qe-gpei-proxy-12160505-m-0.c.openshift-qe.internal 0/1 Completed 0 4h34m
installer-6-qe-gpei-proxy-12160505-m-1.c.openshift-qe.internal 0/1 Completed 0 4h35m
installer-6-qe-gpei-proxy-12160505-m-2.c.openshift-qe.internal 0/1 Completed 0 4h33m
kube-controller-manager-qe-gpei-proxy-12160505-m-0.c.openshift-qe.internal 3/3 Running 0 4h34m
kube-controller-manager-qe-gpei-proxy-12160505-m-1.c.openshift-qe.internal 3/3 Running 0 4h35m
kube-controller-manager-qe-gpei-proxy-12160505-m-2.c.openshift-qe.internal 3/3 Running 0 4h33m
revision-pruner-4-qe-gpei-proxy-12160505-m-1.c.openshift-qe.internal 0/1 Completed 0 4h42m
revision-pruner-5-qe-gpei-proxy-12160505-m-0.c.openshift-qe.internal 0/1 Completed 0 4h35m
revision-pruner-5-qe-gpei-proxy-12160505-m-2.c.openshift-qe.internal 0/1 Completed 0 4h41m
revision-pruner-6-qe-gpei-proxy-12160505-m-0.c.openshift-qe.internal 0/1 Completed 0 4h33m
revision-pruner-6-qe-gpei-proxy-12160505-m-1.c.openshift-qe.internal 0/1 Completed 0 4h34m
revision-pruner-6-qe-gpei-proxy-12160505-m-2.c.openshift-qe.internal 0/1 Completed 0 4h31m
Inside the kcm pod, CA cert for proxy could be found in /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.