Bug 1772756
| Summary: | cloud provider could not be initialized behind mitm proxy which is using some self-signed certificates | |||
|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Johnny Liu <jialiu> | |
| Component: | kube-controller-manager | Assignee: | Maciej Szulik <maszulik> | |
| Status: | CLOSED ERRATA | QA Contact: | Gaoyun Pei <gpei> | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 4.3.0 | CC: | anbhat, aos-bugs, bbennett, dhansen, gpei, mfojtik | |
| Target Milestone: | --- | Keywords: | TestBlocker | |
| Target Release: | 4.4.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | Bug Fix | ||
| Doc Text: |
Cause:
Certificates were not properly propagated.
Consequence:
Cloud provider could not be initialized behind mitm proxy.
Fix:
Properly propagate certificates to kube-controller-manager.
Result:
Cloud provider works as expected with proxy set.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1782819 (view as bug list) | Environment: | ||
| Last Closed: | 2020-05-04 11:15:35 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1782819 | |||
|
Description
Johnny Liu
2019-11-15 05:43:39 UTC
@Daneyon, do we have any update on this issue? googleapi.com is not automatically added to the default no proxy list. It appears the ca trust bundle being used by controllermanager does not include the ca that signed the proxy's cert. Can you confirm that you created the ca cert configmap and that you referenced the configmap in proxy.spec.trustedCA? Reproduced this issue using payload 4.3.0-0.nightly-2019-12-04-214544 No configmap with config.openshift.io/inject-trusted-cabundle="true" label created under openshift-kube-controller-manager namespace. # oc get cm -n openshift-kube-controller-manager -o yaml |grep inject-trusted-cabundle So this looks like some remaining tasks that need to be completed from kube-controller-manager side for fully proxy support. Based on the discussion I've had with mfojtik, and what was stated already in https://bugzilla.redhat.com/show_bug.cgi?id=1772756#c2 it's a network bug, and googleapi.com should be whitelisted in NO_PROXY. Moving this back to networking. KCM should never reach to any external location, except for cloud providers. The controller manager should add trusted ca support so it can proxy https connections. By doing so, cluster admins will have choice whether or not to proxy gcloud api calls. If not, then add https://www.googleapis.com/compute to the default no_proxy lis. Note that product docs will need to be updated to reflect this requirement. Another option is to require https://cloud.google.com/vpc/docs/private-access-options for gce with mitm proxy. Working on a fix... Verify this bug with payload 4.4.0-0.nightly-2019-12-14-103510
# oc get cm trusted-ca-bundle -n openshift-kube-controller-manager -o yaml
...
metadata:
creationTimestamp: "2019-12-16T05:30:13Z"
labels:
config.openshift.io/inject-trusted-cabundle: "true"
name: trusted-ca-bundle
namespace: openshift-kube-controller-manager
resourceVersion: "3745"
# oc get pod -n openshift-kube-controller-manager
NAME READY STATUS RESTARTS AGE
installer-4-qe-gpei-proxy-12160505-m-1.c.openshift-qe.internal 0/1 Completed 0 4h42m
installer-5-qe-gpei-proxy-12160505-m-0.c.openshift-qe.internal 0/1 Completed 0 4h40m
installer-5-qe-gpei-proxy-12160505-m-2.c.openshift-qe.internal 0/1 Completed 0 4h42m
installer-6-qe-gpei-proxy-12160505-m-0.c.openshift-qe.internal 0/1 Completed 0 4h34m
installer-6-qe-gpei-proxy-12160505-m-1.c.openshift-qe.internal 0/1 Completed 0 4h35m
installer-6-qe-gpei-proxy-12160505-m-2.c.openshift-qe.internal 0/1 Completed 0 4h33m
kube-controller-manager-qe-gpei-proxy-12160505-m-0.c.openshift-qe.internal 3/3 Running 0 4h34m
kube-controller-manager-qe-gpei-proxy-12160505-m-1.c.openshift-qe.internal 3/3 Running 0 4h35m
kube-controller-manager-qe-gpei-proxy-12160505-m-2.c.openshift-qe.internal 3/3 Running 0 4h33m
revision-pruner-4-qe-gpei-proxy-12160505-m-1.c.openshift-qe.internal 0/1 Completed 0 4h42m
revision-pruner-5-qe-gpei-proxy-12160505-m-0.c.openshift-qe.internal 0/1 Completed 0 4h35m
revision-pruner-5-qe-gpei-proxy-12160505-m-2.c.openshift-qe.internal 0/1 Completed 0 4h41m
revision-pruner-6-qe-gpei-proxy-12160505-m-0.c.openshift-qe.internal 0/1 Completed 0 4h33m
revision-pruner-6-qe-gpei-proxy-12160505-m-1.c.openshift-qe.internal 0/1 Completed 0 4h34m
revision-pruner-6-qe-gpei-proxy-12160505-m-2.c.openshift-qe.internal 0/1 Completed 0 4h31m
Inside the kcm pod, CA cert for proxy could be found in /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0581 |