Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1802213

Summary: The information from image.config.openshift.io/cluster is not populated correctly in /etc/containers/registries.conf
Product: OpenShift Container Platform Reporter: Jose Ortiz Padilla <jortizpa>
Component: NodeAssignee: Urvashi Mohnani <umohnani>
Status: CLOSED DUPLICATE QA Contact: Sunil Choudhary <schoudha>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.2.zCC: acomabon, andbartl, aos-bugs, dornelas, frank.lamon, jokerman, mpatel, rheinzma, rphillips, rsandu, skrenger, sreber, umohnani
Target Milestone: ---   
Target Release: 4.5.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-21 18:03:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jose Ortiz Padilla 2020-02-12 15:48:52 UTC
Description of problem:
The information in image.config.openshift.io/cluster added under 
allowedRegistriesForImport and allowedRegistries is not being populated in /etc/containers/registries.conf
It is being populated just in /etc/containers/policy.json

Version-Release number of selected component (if applicable):
4.2.14

How reproducible:
Add allowed registries information in image.config.openshift.io/cluster

Steps to Reproduce:
1. Edit image.config.openshift.io/cluster
2. Add a config similar to:
spec:
  additionalTrustedCA:
    name: registry-config
  allowedRegistriesForImport:
  - domainName: myinternal.registry.example.com
    insecure: false
  registrySources:
    allowedRegistries:
    - myinternal.registry.example.com
    - registry.access.redhat.com
    - docker.io

Actual results:
/etc/containers/registries.conf just shows:
unqualified-search-registries = ['registry.access.redhat.com', 'docker.io']

Expected results:
/etc/containers/registries.conf showing something similar to:

unqualified-search-registries = ["registry.access.redhat.com", "docker.io"]

[[registry]]
  location = "docker.io"
  insecure = true
  blocked = false
  mirror-by-digest-only = false
  prefix = ""

[[registry]]
  location = "quay.io"
  insecure = false
  blocked = false
  mirror-by-digest-only = false
  prefix = ""

[[registry]]
  location = "myinternal.registry.example.com"
  insecure = false
  blocked = false
  mirror-by-digest-only = false
  prefix = ""

Additional info:
I have checked that there is some information in policy.json, but it seems not to work.

Comment 9 Urvashi Mohnani 2020-02-19 17:07:13 UTC
So these are the steps for the workaround:

1. Use the ICSP CR to set the allowed registries so it can be whitelisted in the policy.json file
2. Base64 the content the CU wants in the registries.conf - e.g. unqualified-search-registries = ['reg1', 'reg2', 'reg3'] encoded to base 64
3. Create a MC yaml similar to the example here https://github.com/openshift/machine-config-operator#applying-configuration-changes-to-the-cluster. Replace the `source` with the base64 content from step 2.
Note: If the CU wants it on both master and worker nodes, they will have to create 2 of these MCs where `machineconfiguration.openshift.io/role` is set to `worker` in one and `master` in the other.
4. Then run `oc create mc.yaml` and the changes will roll out to the nodes.

And yes, do not edit 'image.config.openshift.io/cluster' as that might overwrite the changes from the workaround above. If the CU doesn't care about whitelisting registries and just want to use short names, then step 1 is not needed.

Comment 12 Urvashi Mohnani 2020-02-26 15:25:11 UTC
yup, done!

Comment 15 Urvashi Mohnani 2020-04-01 17:48:09 UTC
Hi Simon,

We have support for drop-in files for registries.conf now in OCP 4.3.10. If the customer upgrades to this version, they can do the following steps to modify "unqualified-search-registries" to as they please.

1) Upgrade to OCP 4.3.10
2) Create a mc yaml file with the changes you want for "unqualified-search-registries" --> Follow the instructions at https://github.com/openshift/machine-config-operator#applying-configuration-changes-to-the-cluster on how to create the yaml for your custom MC.
   - The way drop-in files for registries.conf works is you create a new file in "/etc/containers/registries.conf.d/". cri-o will pick up files in alpha numerical order. So "/etc/containers/registries.conf.d/02-myconfig.conf" will take priority over "/etc/containers/registries.conf.d/01-firstconfig". In the event you want to read more on this https://github.com/containers/image/blob/master/docs/containers-registries.conf.d.5.md
   - If you want these changes on both master and worker nodes, you will have to create two MC yamls with the "machineconfiguration.openshift.io/role" set to "worker" for one, and "master" for the other.
3) Run "oc create -f <mc.yaml>" and wait for the changes to roll out to the nodes. The changes will not be overwritten if these steps are followed.

Here is an example MC yaml that I used to roll out changes for unqualified-search-registries:

➜  ~ cat reg.yaml 
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: set-unqualified-registries
spec:
  config:
    ignition:
      version: 2.2.0
    storage:
      files:
      - contents:
          source: data:text/plain;charset=utf-8;base64,dW5xdWFsaWZpZWQtc2VhcmNoLXJlZ2lzdHJpZXMgPSBbJ2JsYWguaW8nLCAnZG9ja2VyLmlvJ10K
        filesystem: root
        mode: 0644
        path: /etc/containers/registries.conf.d/01-myregconfig.conf

➜  ~ echo dW5xdWFsaWZpZWQtc2VhcmNoLXJlZ2lzdHJpZXMgPSBbJ2JsYWguaW8nLCAnZG9ja2VyLmlvJ10K | base64 -d
unqualified-search-registries = ['blah.io', 'docker.io']

Comment 17 frank.lamon 2020-04-16 13:59:04 UTC
We upgraded to 4.3.10 and dropping a conf file in /etc/containers/registries.d/ still doesn't work for us.

Still has to be added to /etc/containers/registries.conf to be searchable.

Comment 18 Urvashi Mohnani 2020-04-20 16:11:59 UTC
You need to drop the conf file in /etc/containers/registries.conf.d for this to work. I just verified on a 4.3.10 cluster and it is working as expected.

Comment 19 frank.lamon 2020-04-20 16:25:26 UTC
We tried creating the /etc/containers/registries.conf.d directory and dropping the conf file in there as well.  The nodes are all coreos and have crio version 1.16.4-1.dev.rhaos4.3.git9238eee.el8.

We're going to go from 4.3.12 to 4.3.13 later today and see if this changes.