Description of problem:
When mentioned the variable "openshift_docker_blocked_registries" in the inventory with 'all' value assigned(while doing fresh installation), it's not blocking the registries at all.
As per the usage of this variable, it should block the registries except for the allowed registries. But still, the registries.block section in the /etc/containers/registries.conf is empty.
1) Followed Host preparation guide: https://docs.openshift.com/container-platform/3.11/install/host_preparation.html
2) Ran the prerequisites.yaml playbook from /usr/share/ansible/openshift-ansible/playbooks/prerequisites.yml
3) Ran the deploy_cluster.yml from /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml
Version-Release number of the following components:
rpm -q openshift-ansible - openshift-ansible-3.11.69-1.git.0.2ff281f.el7.noarch
rpm -q ansible - ansible-2.6.12-1.el7ae.noarch
ansible --version - ansible 2.6.12
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/home/ocp-mgmt/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /bin/ansible
python version = 2.7.5 (default, Sep 12 2018, 05:31:16) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
Steps to Reproduce:
1. Mention openshift_docker_blocked_registries parameter in the registry with value "all"
2. Run the prerequisites.yml, deploy_cluster.yaml
3. After installation check, if registry.redhat.io or other registries are accessible or not which are not included in the openshift_docker_additional_registries
Registries are accessible after blocking.
Please include the entire output from the last TASK line through the end of output if an error is generated
Please attach logs from ansible-playbook with the -vvv flag
Please, whenever possible, include actual inputs and outputs rather than describing them which is open to misinterpretation.
Reviewing the code this will populate the values into /etc/sysconfig/docker by default rather than /etc/containers/registries.conf. Can you confirm that's the case and that the settings defined therein are taking effect?
The output of `docker info` will illustrate how exactly docker is configured.
(In reply to Scott Dodson from comment #1)
> Please, whenever possible, include actual inputs and outputs rather than
> describing them which is open to misinterpretation.
> Reviewing the code this will populate the values into /etc/sysconfig/docker
> by default rather than /etc/containers/registries.conf. Can you confirm
> that's the case and that the settings defined therein are taking effect?
> The output of `docker info` will illustrate how exactly docker is configured.
Yes, we can confirm that's the case.
--add-registry registry.redhat.io <--- This value should not be there in ADD_REGISTRY, after assigning "all" to "openshift_docker_blocked_registries" this variable.
In this case, it's assigned in the below manner.
ADD_REGISTRY='--add-registry <internal-artifactory> --add-registry docker-registry.default.svc:5000 --add-registry registry.redhat.io'
Ok, we append that for all installs where openshift_deployment_type == 'openshift-enterprise'. I guess it makes sense to only do this when (openshift_deployment_type == 'openshift-enterprise' && oreg_url matches 'registry.redhat.io').
Can you please set an appropriate severity/priority based on your customer's opinion of this matter? It will then be worked on based on PM Score ordering.
Submitted PR: https://github.com/openshift/openshift-ansible/pull/11390
1. The PR is not good. https://github.com/openshift/openshift-ansible/pull/11390
oreg_url is optional,
if oreg_url is not set and openshift_docker_blocked_registries=all
install will fail as the registry is blocked to pull images
2. I think the logic would be something like:
all registries other than (listed in openshift_docker_additional_registries plus registry in oreg_url) are blocked.
if oreg_url not set and openshift_deployment_type=openshift-enterprise, default registry(registry.redhat.io) is allowed, unless explicitily listed in openshift_docker_blocked_registries.
"Jun 17 05:13:57 ip-172-18-8-21.ec2.internal atomic-openshift-node: E0617 05:13:57.028197 13414 kuberuntime_manager.go:646] createPodSandbox for pod \"master-api-ip-172-18-8-21.ec2.internal_kube-system(9d066f84b20195c767ec4ed9d7ac3ba2)\" failed: rpc error: code = Unknown desc = All endpoints blocked.",
[root@ip-172-18-8-21 ~]# cat /etc/containers/registries.conf
# Ansible managed
# This is a system-wide configuration file used to
# keep track of registries for various container backends.
# It adheres to TOML format and does not support recursive
# lists of registries.
# The default location for this configuration file is /etc/containers/registries.conf.
# The only valid categories are: 'registries.search', 'registries.insecure',
# and 'registries.block'.
registries = 
# If you need to access insecure registries, add the registry's fully-qualified name.
# An insecure registry is one that does not have a valid SSL certificate or only does HTTP.
registries = ["brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888", "virt-openshift-05.lab.eng.nay.redhat.com:5000", "virt-openshift-05.lab.eng.nay.redhat.com:5001", "registry.reg-aws.openshift.com:443", "asb-registry.usersys.redhat.com:5000"]
# If you need to block pull access from a registry, uncomment the section below
# and add the registries fully-qualified name.
# Docker only
registries = ["all"]
[root@ip-172-18-8-21 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
[root@ip-172-18-8-21 ~]# docker pull asd
Using default tag: latest
Error response from daemon: No configured registry to pull from.
[root@ip-172-18-8-21 ~]# docker pull registry.reg-aws.openshift.com:443/openshift3/ose-node:v3.11
Trying to pull repository registry.reg-aws.openshift.com:443/openshift3/ose-node ...
All endpoints blocked.
Be aware that when openshift_docker_blocked_registries=all set,
if oreg_url is specified, must set the registry used in oreg_url to openshift_docker_additional_registries, or install will fail due to registry blocked.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.