Bug 1866268

Description of problem:

My customer has an issue when using both sigstore and whitelists together with the MCO

We tried implementing a sigstore using  https://access.redhat.com/verify-images-ocp4 and we do not lose our whitelist registry changes. But also cannot change policy.json to allow for image signature verification as described in the doc above. Looks like the MCO (or other operator?, had no luck finding the source code responsible for this) also dynamically creates a MachineConfig object for /etc/containers/policy.json based on the contents of the image.config.openshift.io/cluster  resource (containing our whitelisting stuff), but with the always winning priority of 99. :)
> oc get  machineconfigpool/worker -o yaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
....
spec:
  configuration:
    name: rendered-worker-f40af94b2551f0c7613030f1f99c0af9
    source:
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 00-worker
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 01-worker-container-runtime
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 01-worker-kubelet
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 51-worker-registry-trust                                    <- contains our new policy.json
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 99-worker-862ae372-453a-4205-926d-abf8dea11270-registries   <- contains generated policy.json
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 99-worker-mtu
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: 99-worker-ssh
    - apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      name: worker-chrony-configuration
> oc get machineconfig 51-worker-registry-trust -o yaml | grep path:
        path: /etc/containers/registries.d/registry.redhat.io.yaml
        path: /etc/containers/registries.d/registry.access.redhat.com.yaml
        path: /etc/pki/<env>/openshift-virtual.<host>.pub
        path: /etc/containers/registries.d/openshift-virtual.<host>.yaml
        path: /etc/containers/policy.json
> oc get machineconfig 99-worker-862ae372-453a-4205-926d-abf8dea11270-registries -o yaml | grep path:
        path: /etc/containers/policy.json
Tested this by doing a manual change to image.config.openshift.io/cluster. The new entry to the spec.registrySources.allowedRegistries popped up in the generated MachineConfig and policy.json on the workers as a new registry. So looks like there is no way to win this... :)


Ideally, we would like to set the sigstore and the whitelist through the same config option.

It should be noted that this function is available in OCP 3.11, so why cant OCP 4.x do this?


How reproducible:
Everytime

Steps to Reproduce:
1.
2.
3.

Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag