Description of problem: Meet "lookup image-registry.openshift-image-registry.svc on 10.196.0.13:53: no such host" when build in an OSP13 cluster which is enabled Kuryr: Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled <unknown> default-scheduler Successfully assigned wzheng2/ruby-ex-3-4qkkq to ostest-2sf2t-worker-5r9z9 Normal Pulling 12s (x2 over 23s) kubelet, ostest-2sf2t-worker-5r9z9 Pulling image "image-registry.openshift-image-registry.svc:5000/wzheng2/ruby-ex@sha256:beee8628f25b9a5e9d1dd54caee69c1e1ffe8cb29ba5cf8d3f402d88650d7903" Warning Failed 12s (x2 over 23s) kubelet, ostest-2sf2t-worker-5r9z9 Failed to pull image "image-registry.openshift-image-registry.svc:5000/wzheng2/ruby-ex@sha256:beee8628f25b9a5e9d1dd54caee69c1e1ffe8cb29ba5cf8d3f402d88650d7903": rpc error: code = Unknown desc = error pinging docker registry image-registry.openshift-image-registry.svc:5000: Get https://image-registry.openshift-image-registry.svc:5000/v2/: dial tcp: lookup image-registry.openshift-image-registry.svc on 10.196.0.29:53: no such host Warning Failed 12s (x2 over 23s) kubelet, ostest-2sf2t-worker-5r9z9 Error: ErrImagePull Normal BackOff 0s (x3 over 23s) kubelet, ostest-2sf2t-worker-5r9z9 Back-off pulling image "image-registry.openshift-image-registry.svc:5000/wzheng2/ruby-ex@sha256:beee8628f25b9a5e9d1dd54caee69c1e1ffe8cb29ba5cf8d3f402d88650d7903" Warning Failed 0s (x3 over 23s) kubelet, ostest-2sf2t-worker-5r9z9 Error: ImagePullBackOff Version-Release number of selected component (if applicable): 4.3.0-0.nightly-2019-12-22-054650 How reproducible: Always Steps to Reproduce: 1.Set up a OSP 13 cluster and enable kuryr 2.Create a build 3.Watch pod status Actual results: $ oc get pods NAME READY STATUS RESTARTS AGE ruby-ex-1-build 0/1 Completed 0 24m ruby-ex-1-4qkkq 0/1 ImagePullBackOff 0 17s ruby-ex-1-deploy 1/1 Running 0 33s Expected results: Pod should be running Additional info: There is a workaround to fix the issue by adding image registry internal address to /etc/hosts in each master/workder： $ oc debug nodes/ostest-2sf2t-master-2 Starting pod/ostest-2sf2t-master-2-debug ... To use host binaries, run `chroot /host` Pod IP: 10.196.0.30 If you don't see a command prompt, try pressing enter. sh-4.2# chroot /host sh-4.4# echo "172.30.179.121 image-registry.openshift-image-registry.svc image-registry.openshift-image-registry.svc.cluster.local # openshift-generated-node-resolver" >> /etc/hosts sh-4.4# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 172.30.97.23 image-registry.openshift-image-registry.svc image-registry.openshift-image-registry.svc.cluster.local # openshift-generated-node-resolver
Verified in 4.4.0-0.nightly-2020-02-10-234204 on top of OSP 13 2020-01-15.3 puddle. Build creation and image pull from internal registry works. $ oc new-app centos/ruby-25-centos7~https://github.com/sclorg/ruby-ex.git $ oc get pods NAME READY STATUS RESTARTS AGE ruby-ex-1-build 0/1 Completed 0 6m22s ruby-ex-1-cqw5h 1/1 Running 0 4m55s ruby-ex-1-deploy 0/1 Completed 0 5m2s $ oc describe pod ruby-ex-1-cqw5h ... Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled <unknown> default-scheduler Successfully assigned test2/ruby-ex-1-cqw5h to ostest-rgz26-worker-9sqkc Normal Pulling 2m33s kubelet, ostest-rgz26-worker-9sqkc Pulling image "image-registry.openshift-image-registry.svc:5000/test2/ruby-ex@sha256:47f28e7b3b6031847f1ac44722e8b2688f7ebf503e0107e0537e11c36d2d7afd" Normal Pulled 2m17s kubelet, ostest-rgz26-worker-9sqkc Successfully pulled image "image-registry.openshift-image-registry.svc:5000/test2/ruby-ex@sha256:47f28e7b3b6031847f1ac44722e8b2688f7ebf503e0107e0537e11c36d2d7afd" Normal Created 2m16s kubelet, ostest-rgz26-worker-9sqkc Created container ruby-ex Normal Started 2m16s kubelet, ostest-rgz26-worker-9sqkc Started container ruby-ex
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0581