Description of problem: When mentioned the variable "openshift_docker_blocked_registries" in the inventory with 'all' value assigned(while doing fresh installation), it's not blocking the registries at all. As per the usage of this variable, it should block the registries except for the allowed registries. But still, the registries.block section in the /etc/containers/registries.conf is empty. Installation steps: 1) Followed Host preparation guide: https://docs.openshift.com/container-platform/3.11/install/host_preparation.html 2) Ran the prerequisites.yaml playbook from /usr/share/ansible/openshift-ansible/playbooks/prerequisites.yml 3) Ran the deploy_cluster.yml from /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml Version-Release number of the following components: rpm -q openshift-ansible - openshift-ansible-3.11.69-1.git.0.2ff281f.el7.noarch rpm -q ansible - ansible-2.6.12-1.el7ae.noarch ansible --version - ansible 2.6.12 config file = /etc/ansible/ansible.cfg configured module search path = [u'/home/ocp-mgmt/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible executable location = /bin/ansible python version = 2.7.5 (default, Sep 12 2018, 05:31:16) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] How reproducible: Always Steps to Reproduce: 1. Mention openshift_docker_blocked_registries parameter in the registry with value "all" 2. Run the prerequisites.yml, deploy_cluster.yaml 3. After installation check, if registry.redhat.io or other registries are accessible or not which are not included in the openshift_docker_additional_registries Actual results: Registries are accessible after blocking. Please include the entire output from the last TASK line through the end of output if an error is generated Expected results: Additional info: Please attach logs from ansible-playbook with the -vvv flag
Please, whenever possible, include actual inputs and outputs rather than describing them which is open to misinterpretation. Reviewing the code this will populate the values into /etc/sysconfig/docker by default rather than /etc/containers/registries.conf. Can you confirm that's the case and that the settings defined therein are taking effect? The output of `docker info` will illustrate how exactly docker is configured.
Hello, (In reply to Scott Dodson from comment #1) > Please, whenever possible, include actual inputs and outputs rather than > describing them which is open to misinterpretation. > > Reviewing the code this will populate the values into /etc/sysconfig/docker > by default rather than /etc/containers/registries.conf. Can you confirm > that's the case and that the settings defined therein are taking effect? > The output of `docker info` will illustrate how exactly docker is configured. Yes, we can confirm that's the case. --add-registry registry.redhat.io <--- This value should not be there in ADD_REGISTRY, after assigning "all" to "openshift_docker_blocked_registries" this variable. In this case, it's assigned in the below manner. ~~~~ ADD_REGISTRY='--add-registry <internal-artifactory> --add-registry docker-registry.default.svc:5000 --add-registry registry.redhat.io' BLOCK_REGISTRY='--block-registry all' INSECURE_REGISTRY='--insecure-registry <internal-artifactory>' ~~~~ Thank you
Ok, we append that for all installs where openshift_deployment_type == 'openshift-enterprise'. I guess it makes sense to only do this when (openshift_deployment_type == 'openshift-enterprise' && oreg_url matches 'registry.redhat.io'). Can you please set an appropriate severity/priority based on your customer's opinion of this matter? It will then be worked on based on PM Score ordering.
Submitted PR: https://github.com/openshift/openshift-ansible/pull/11390
1. The PR is not good. https://github.com/openshift/openshift-ansible/pull/11390 oreg_url is optional, for installation, if oreg_url is not set and openshift_docker_blocked_registries=all install will fail as the registry is blocked to pull images 2. I think the logic would be something like: when openshift_docker_blocked_registries=all, all registries other than (listed in openshift_docker_additional_registries plus registry in oreg_url) are blocked. if oreg_url not set and openshift_deployment_type=openshift-enterprise, default registry(registry.redhat.io) is allowed, unless explicitily listed in openshift_docker_blocked_registries.
PR: https://github.com/openshift/openshift-ansible/pull/11565 Build: openshift-ansible-3.11.112-1
Failed. openshift-ansible-3.11.119-1.git.0.c9a8ebf.el7 oreg_url=registry.reg-aws.openshift.com:443/openshift3/ose-${component}:${version} openshift_docker_insecure_registries=brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888,virt-openshift-05.lab.eng.nay.redhat.com:5000,virt-openshift-05.lab.eng.nay.redhat.com:5001,registry.reg-aws.openshift.com:443,asb-registry.usersys.redhat.com:5000 openshift_docker_blocked_registries=all "Jun 17 05:13:57 ip-172-18-8-21.ec2.internal atomic-openshift-node[13414]: E0617 05:13:57.028197 13414 kuberuntime_manager.go:646] createPodSandbox for pod \"master-api-ip-172-18-8-21.ec2.internal_kube-system(9d066f84b20195c767ec4ed9d7ac3ba2)\" failed: rpc error: code = Unknown desc = All endpoints blocked.", [root@ip-172-18-8-21 ~]# cat /etc/containers/registries.conf # Ansible managed # This is a system-wide configuration file used to # keep track of registries for various container backends. # It adheres to TOML format and does not support recursive # lists of registries. # The default location for this configuration file is /etc/containers/registries.conf. # The only valid categories are: 'registries.search', 'registries.insecure', # and 'registries.block'. [registries.search] registries = [] # If you need to access insecure registries, add the registry's fully-qualified name. # An insecure registry is one that does not have a valid SSL certificate or only does HTTP. [registries.insecure] registries = ["brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888", "virt-openshift-05.lab.eng.nay.redhat.com:5000", "virt-openshift-05.lab.eng.nay.redhat.com:5001", "registry.reg-aws.openshift.com:443", "asb-registry.usersys.redhat.com:5000"] # If you need to block pull access from a registry, uncomment the section below # and add the registries fully-qualified name. # # Docker only [registries.block] registries = ["all"] [root@ip-172-18-8-21 ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE [root@ip-172-18-8-21 ~]# docker pull asd Using default tag: latest Error response from daemon: No configured registry to pull from. [root@ip-172-18-8-21 ~]# docker pull registry.reg-aws.openshift.com:443/openshift3/ose-node:v3.11 Trying to pull repository registry.reg-aws.openshift.com:443/openshift3/ose-node ... All endpoints blocked.
Fixed. openshift-ansible-3.11.119-1.git.0.c9a8ebf.el7 Be aware that when openshift_docker_blocked_registries=all set, if oreg_url is specified, must set the registry used in oreg_url to openshift_docker_additional_registries, or install will fail due to registry blocked.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:1605