Description of problem: Pod image is pulled by kubelet and it should work similar to docker pull. When relative image path is used, 'docker pull' searches the image in all the registries and is able to download the image but kubelet fails to download the image as it is only searching from docker.io registry. Looks like some mis-configuration? Version-Release number of selected component (if applicable): [root@ip-172-18-1-193 ~]# oc version oc v3.10.0-0.54.0 kubernetes v1.10.0+b81c8f8 How reproducible: Always Steps to Reproduce: 1. Launch openshift on AWS using ansible installer 2. Create sample pod with relative image path (eg: openshift3/ose-deployer) 3. Pod creation fails with error 'container start failed: ErrImagePull: rpc error: code = Unknown desc = repository docker.io/openshift3/ose-deployer not found: does not exist or no pull access' 4. docker pull openshift3/ose-deployer works, downloads the image from registry.reg-aws.openshift.com:443/openshift3/ose-deployer Actual results: Pod fails with image pull error Expected results: Pod creation should succeed Additional info:
Tried adding AWS registry to /etc/containers/registries.conf, restarted docker and atomic-openshift-node service but that did not resolve the issue. More details from openshift node: [root@ip-172-18-3-185 ~]# cat /etc/containers/registries.conf ... [registries.search] registries = ['registry.access.redhat.com', 'registry.reg-aws.openshift.com:443'] ... [registries.insecure] registries = ['registry.reg-aws.openshift.com:443'] ------------- [root@ip-172-18-3-185 ~]# docker info ... Insecure Registries: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888 registry.reg-aws.openshift.com:443 virt-openshift-05.lab.eng.nay.redhat.com:5000 virt-openshift-05.lab.eng.nay.redhat.com:5001 asb-registry.usersys.redhat.com:5000 127.0.0.0/8 ... Registries: registry.reg-aws.openshift.com:443 (insecure), registry.access.redhat.com (secure), registry.access.redhat.com (secure), registry.reg-aws.openshift.com:443 (insecure), docker.io (secure) --------------- [root@ip-172-18-3-185 ~]# cat /run/containers/registries.conf REGISTRIES="--add-registry registry.access.redhat.com --add-registry registry.reg-aws.openshift.com:443 --insecure-registry registry.reg-aws.openshift.com:443
This is a change in behavior of 3.10. You need to use fully qualified image references for your pods as far as I know but moving to Image component to verify this.
The image references the kubelet uses is not the domain of the Image component.
I think you need to do: systemctl restart registries after changing /etc/containers/registries.conf
I'll validate this with @weliang and see.
*** This bug has been marked as a duplicate of bug 1583500 ***