Description of problem: It's no longer possible to make inter pod API calls to openshift.default.svc with /var/run/secrets/kubernetes.io/serviceaccount/ca.crt file. Our python script in pod is getting following error: CRITICAL:__main__:Cannot query OpenShift API for "https://openshift.default.svc/api/v1/namespaces/mnovak-mynamespace/pods" Traceback (most recent call last): File "/opt/partition/queryosapi.py", line 151, in <module> queryResult = getLivingPods() File "/opt/partition/queryosapi.py", line 109, in getLivingPods jsonPodsData = getPodsJsonData() File "/opt/partition/queryosapi.py", line 97, in getPodsJsonData jsonText = OpenShiftQuery.queryApi('/api/v1/namespaces/{}/pods'.format(OpenShiftQuery.getNameSpace())) File "/opt/partition/queryosapi.py", line 89, in queryApi return urllib2.urlopen(request, cafile = OpenShiftQuery.CERT_FILE_PATH).read() File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen return opener.open(url, data, timeout) File "/usr/lib64/python2.7/urllib2.py", line 431, in open response = self._open(req, data) File "/usr/lib64/python2.7/urllib2.py", line 449, in _open '_open', req) File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/usr/lib64/python2.7/urllib2.py", line 1258, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib64/python2.7/urllib2.py", line 1211, in do_open h.request(req.get_method(), req.get_selector(), req.data, headers) File "/usr/lib64/python2.7/httplib.py", line 1041, in request self._send_request(method, url, body, headers) File "/usr/lib64/python2.7/httplib.py", line 1075, in _send_request self.endheaders(body) File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders self._send_output(message_body) File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output self.send(msg) File "/usr/lib64/python2.7/httplib.py", line 843, in send self.connect() File "/usr/lib64/python2.7/httplib.py", line 1263, in connect ssl.match_hostname(self.sock.getpeercert(), server_hostname) File "/usr/lib64/python2.7/ssl.py", line 267, in match_hostname % (hostname, ', '.join(map(repr, dnsnames)))) ssl.CertificateError: hostname 'openshift.default.svc' doesn't match either of 'kubernetes', 'kubernetes.default', 'kubernetes.default.svc', 'kubernetes.default.svc.cluster.local', '172.30.0.1' This seems to be change against OCP 3.11 where it works. Version-Release number of selected component (if applicable): 4.1.0-rc.3 How reproducible - always: Run from inside a pod: curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt -H "Authorization: Bearer dBNsUYunIES1nIB0yfL-NNSE51Nk_cRXcDSzKz2dEuo" 'https://openshift.default.svc/api/v1/namespaces/mnovak-mynamespace/pods'
I assume problem will be that certificate for kube api server does not contains `openshift.default.svc` as Subject Alternative name. curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt https://openshift.default.svc curl: (51) Unable to communicate securely with peer: requested domain name does not match the server's certificate.
Fix: https://github.com/openshift/installer/pull/1750
Additionally: https://github.com/openshift/cluster-kube-apiserver-operator/pull/473
> It's no longer possible to make inter pod API calls to openshift.default.svc with /var/run/secrets/kubernetes.io/serviceaccount/ca.crt file. Our python script in pod is getting following error: What's the impact of this? It's not my call, but I'm trying to understand if this should block 4.1.0, or if it's ok to land in 4.1.z, or if we can skip it for 4.1 entirely.
openshift.default.svc has been in the OpenShift docs in a couple of places for a long time, for example https://docs.openshift.com/container-platform/3.11/dev_guide/service_accounts.html#using-a-service-accounts-credentials-inside-a-container and https://docs.openshift.com/container-platform/3.11/architecture/additional_concepts/authentication.html#oauth-server-metadata So I think as it was documented behavior, it should block 4.1.0.
This bug targets 4.1.0 (comment 7 makes sense to me, but again, not my call), and [1] is still open. Back to POST. [1]: https://github.com/openshift/installer/pull/1751
4.1 is merged, moving to QA.
Verified in latest payload 4.1.0-0.nightly-2019-05-18-050636 env: $ oc rsh mysql-1-2dlfh sh-4.2$ curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt -H "Authorization: Bearer cuHZWeyX..." 'https://openshift.d efault.svc/api/v1/namespaces/testuser-37-proj/pods' { "kind": "PodList", "apiVersion": "v1", "metadata": { ...
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0758