Description of problem: When installing or upgrading to OCP 3.9.30, the kubelet is trying to pull ose-pod from docker.io, resulting in error in every deployment. This is an installation without oreg_url, so it should default every internal component pull to registry.access.redhat.com Also, during installation/upgrade the webconsole tries to deploy using that docker.io/openshift3/ose-pod image, which causes the deploy to fail and ending the playbook: Warning FailedCreatePodSandBox 15s (x16 over 4m) kubelet, master-0.mordor.quicklab.pnq2.cee.redhat.com Failed create pod sandbox: rpc error: code = Unknown desc = failed pulling image "openshift3/ose-pod:v3.9.30": repository docker.io/openshift3/ose-pod not found: does not exist or no pull access - node-config looks like this: imageConfig: format: openshift3/ose-${component}:${version} latest: False - /etc/sysconfig/docker: ADD_REGISTRY='--add-registry registry.access.redhat.com' - /etc/containers/registries.conf: [registries.search] registries = ['registry.access.redhat.com'] But, for some reason the deploy always try to pull ose-pod from docker.io The workaround is to tune in every node the imageConfig in /etc/origin/node/node-config.yaml, to this: imageConfig: format: registry.access.redhat.com/openshift3/ose-${component}:${version} latest: False After that and forcing the registry for internal components everything works fine. Version-Release number of the following components: 3.9.30 How reproducible: Steps to Reproduce: 1. upgrade an existing env or install a fresh 3.9.30 Actual results: Error pulling the image, because it doesn't exist. Expected results: Internal componentes pulled by default from registry.access.redhat.com
I cannot reproduce this. Please provide docker logs from the docker daemon experiencing the error or grant us access to the test host you've reproduced this on. Please advise the customer to use the workaround of configuring the fully qualified format string in the mean time. That also needs to make it into /etc/origin/master/master-config.yaml Setting oreg_url to the same string would achieve the same outcome.
I am experiencing the exact same issue: Failed create pod sandbox: rpc error: code = Unknown desc = failed pulling image \"openshift3/ose-pod:v3.9.30\": repository docker.io/openshift3/ose-pod not found: does not exist or no pull access However, when using the command 'docker pull openshift3/ose-pod' manually, it does pull the docker image, however not from docker.io but from registry.access.redhat.com, so the error is a little misleading. When using 'docker pull ose-pod', I do get the error which I am experiencing when installing Openshift: docker pull ose-pod Using default tag: latest Trying to pull repository registry.access.redhat.com/ose-pod ... Trying to pull repository registry.access.redhat.com/ose-pod ... Trying to pull repository docker.io/library/ose-pod ... repository docker.io/ose-pod not found: does not exist or no pull access I think you just need to add openshift3 to the pull command, to make it 'docker pull openshift3/ose-pod', that will solve the issue. Please update ASAP, as at this point it is impossible to deploy a new cluster.
Facing this issue as well. No additional useful logs to contribute, the symptoms are exactly the same for us. The deployment image reference is properly prefixed with openshift3/ose-pod:v3.9.30 and the docker process has --add-registry registry.access.redhat.com yet the logs show it can't pull from docker.io/openshift3/ose-pod:v3.9.30.
mind https://access.redhat.com/solutions/3480921