Bug 1667824
Summary: | Pod created from registry with self-certificate cannot be running with 509 error | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | Wenjing Zheng <wzheng> | ||||||||
Component: | Image Registry | Assignee: | Ben Parees <bparees> | ||||||||
Status: | CLOSED ERRATA | QA Contact: | Wenjing Zheng <wzheng> | ||||||||
Severity: | medium | Docs Contact: | |||||||||
Priority: | medium | ||||||||||
Version: | 4.1.0 | CC: | aos-bugs | ||||||||
Target Milestone: | --- | ||||||||||
Target Release: | 4.1.0 | ||||||||||
Hardware: | Unspecified | ||||||||||
OS: | Unspecified | ||||||||||
Whiteboard: | |||||||||||
Fixed In Version: | Doc Type: | No Doc Update | |||||||||
Doc Text: |
undefined
|
Story Points: | --- | ||||||||
Clone Of: | Environment: | ||||||||||
Last Closed: | 2019-06-04 10:42:06 UTC | Type: | Bug | ||||||||
Regression: | --- | Mount Type: | --- | ||||||||
Documentation: | --- | CRM: | |||||||||
Verified Versions: | Category: | --- | |||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||
Embargoed: | |||||||||||
Attachments: |
|
Description
Wenjing Zheng
2019-01-21 08:53:35 UTC
Please provide the output of: oc get ds -o node-ca -o yaml -n openshift-image-registry oc get cm -o yaml -n openshift-image-registry and then on your nodes, "tree /etc/docker/certs.d" fyi the build image pull won't work until https://jira.coreos.com/browse/DEVEXP-154 is complete, so that is not a bug right now. However pods should be able to pull images from a registry if the CAs are setup properly, the information i requested in comment 1 should help determine what is happening there. Created attachment 1522305 [details]
oc get cm -o yaml -n openshift-image-registry
Created attachment 1522306 [details]
oc get ds -o node-ca -o yaml -n openshift-image-registry
Output of /etc/docker/certs.d from my node: [root@ip-10-0-155-184 ~]# ls /etc/docker/certs.d/ca.crt/ ca.crt [root@ip-10-0-155-184 ~]# cat /etc/docker/certs.d/ca.crt/ca.crt -----BEGIN CERTIFICATE----- MIIC6jCCAdKgAwIBAgIBATANBgkqhkiG9w0BAQsFADAmMSQwIgYDVQQDDBtvcGVu c2hpZnQtc2lnbmVyQDE1NDgxMjI1NjQwHhcNMTkwMTIyMDIwMjQ0WhcNMjQwMTIx MDIwMjQ1WjAmMSQwIgYDVQQDDBtvcGVuc2hpZnQtc2lnbmVyQDE1NDgxMjI1NjQw ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDDaNWKPWXn/yB+4RrDb0t4 7wjbGLk4GPwEkqtQArmNN8anbn6wdXdKySfjnlYYCWtp/T+LIWm4DShwWjH55yXq 8eD4lgDJQOhftbshpEyb42kd95bZi+hTpVdVUan6pG3WLvQSB1Ho9LAyAc6ha2KE QvMe1D4PvrF0eOsIwrWHCGuAKQxCJ1yV030GADK984nXcYxQMDG2NZpyZSvfkJnz k06HvobC7598DVGvjmoup0lg6QcTfOWBaYdXGDg3cFzapnq7clZOLX1DyztbJz0+ ksrpDovmKSktoNQcrr/wpd+3jkp8D91gS5lZ+/O6jteZVHQ1NbY+i/RcCl1DRuJ/ AgMBAAGjIzAhMA4GA1UdDwEB/wQEAwICpDAPBgNVHRMBAf8EBTADAQH/MA0GCSqG SIb3DQEBCwUAA4IBAQA/MRheOL1ViVZYTase4SNlWpFwO7zbzEvJjgi3TjK0s1r+ sl050bccdNelnMwJS3P96HYpHwbrn3aI5hvdgVn+QuVaFon2ETryIWmq/HHED+Iu nssQICtpX3XhbkgLEEOpW78UiTSzDyRjLBKGucvfu+a8EbiWcT0hSNz9DGcPO36C +mmPBYoaZl4eijYdfZ8wZ+YcabrSCWVKnP0ynvg8zhNQYgvL1IEqTf2LKjO1Q1jO LfsJI3Et16cSvvKsaOjDbb1t7XRtPEXN9RPWUv7YTY9ZL5j/ugCc9eg6eN6b+gwg HEss8zlkwNeThJoIByNJmGg4qpc3YZAbFJj3j6VQ -----END CERTIFICATE----- This is my test file about creating a pod: apiVersion: v1 kind: Pod metadata: name: my-pod labels: app: my-pod spec: containers: - name: busybox image: docker-registry-default.apps.0122-47n.qe.rhcloud.com/test/myimage command: ["sh", "-c", "while true; do sleep 10; done"] ports: - containerPort: 80 > cat /etc/docker/certs.d/ca.crt/ca.crt
this is the issue. The cert should be in a directory named "docker-registry-default.apps.0122-47n.qe.rhcloud.com". Since you named your key in the configmap "ca.crt" it is not.
You need to update your configmap, the key in the configmap needs to be the name of the registry host. In addition if you need to specify a port, you need to define the key using ".." instead of ":" (because : is not a valid character for a configmap key):
for example:
host.domain.com..5000
Thanks, Ben! Verified with below version: $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.0.0-0.nightly-2019-01-25-205123 True False 1h Cluster version is 4.0.0-0.nightly-2019-01-25-205123 [root@ip-10-0-135-86 ~]# ls /etc/docker/certs.d/ docker-registry-default.apps.0128-49c.qe.rhcloud.com image-registry.openshift-image-registry.svc:5000 image-registry.openshift-image-registry.svc.cluster.local:5000 [root@ip-10-0-135-86 ~]# ls /etc/docker/certs.d/docker-registry-default.apps.0128-49c.qe.rhcloud.com ca.crt [wzheng@laptop test]$ oc get pods NAME READY STATUS RESTARTS AGE my-pod 1/1 Running 0 34s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 3m default-scheduler Successfully assigned wzheng1/my-pod to ip-10-0-154-255.us-east-2.compute.internal Normal Pulling 3m kubelet, ip-10-0-154-255.us-east-2.compute.internal pulling image "docker-registry-default.apps.0128-49c.qe.rhcloud.com/test/myimage" Normal Pulled 3m kubelet, ip-10-0-154-255.us-east-2.compute.internal Successfully pulled image "docker-registry-default.apps.0128-49c.qe.rhcloud.com/test/myimage" Normal Created 3m kubelet, ip-10-0-154-255.us-east-2.compute.internal Created container Normal Started 3m kubelet, ip-10-0-154-255.us-east-2.compute.internal Started container [wzheng@laptop test]$ cat busybox.yml apiVersion: v1 kind: Pod metadata: name: my-pod labels: app: my-pod spec: containers: - name: busybox image: docker-registry-default.apps.0128-49c.qe.rhcloud.com/test/myimage command: ["sh", "-c", "while true; do sleep 10; done"] ports: - containerPort: 80 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0758 |