Description of problem: ---------------------- After successfully adding the dataImportCronTemplates data to HCO CR, SSP CR is reconciled endlessly. Alternately, it becomes ready (Available: True, Progressing: False, etc.) and back to not ready every couple of seconds. The ssp operator pod logs show the reconciliation error repeatedly: 2021-12-30T23:22:10.443Z ERROR controller-runtime.manager.controller.ssp Reconciler error {"reconciler group": "ssp.kubevirt.io", "reconciler kind": "SSP", "name": "ssp-kubevirt-hyperconverged", "namespace": "openshift-cnv", "error": "Operation cannot be fulfilled on ssps.ssp.kubevirt.io \"ssp-kubevirt-hyperconverged\": the object has been modified; please apply your changes to the latest version and try again"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:253 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:214 Version-Release number of selected component (if applicable): ------------------------------------------------------------ 4.10.0-545 (pre-release) How reproducible: ---------------- 100% Steps to Reproduce: ------------------ 1. oc patch hco kubevirt-hyperconverged -n openshift-cnv --type merge -p '{"spec": {"featureGates": {"enableCommonBootImageImport": true}}}' 2. oc edit hco -n openshift-cnv 3. add the following entry under the spec: dataImportCronTemplates: - metadata: annotations: cdi.kubevirt.io/storage.bind.immediate.requested: "true" name: my-image-cron spec: garbageCollect: Outdated managedDataSource: my schedule: 59 23/3 * * * template: metadata: {} spec: source: registry: imageStream: my-guest pullMethod: node storage: resources: requests: storage: 10Gi status: {} 4. watch the ssp operator pod logs: oc logs --follow ssp-operator-64b46b479b-2txj8 -n openshift-cnv --tail 1 Actual results: -------------- 1. repeatedly reconciliation errors in ssp operator pod logs: 2021-12-30T23:22:10.443Z ERROR controller-runtime.manager.controller.ssp Reconciler error {"reconciler group": "ssp.kubevirt.io", "reconciler kind": "SSP", "name": "ssp-kubevirt-hyperconverged", "namespace": "openshift-cnv", "error": "Operation cannot be fulfilled on ssps.ssp.kubevirt.io \"ssp-kubevirt-hyperconverged\": the object has been modified; please apply your changes to the latest version and try again"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:253 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:214 2. ssp status conditions alternately switches between ready conditions (Available: True, Progressing: False, Degraded: False) and not ready conditions (Available: False, Progressing: True, Degraded: True) every now and then (ready conditions for 1-2 seconds and then to not ready for a longer period) Expected results: ---------------- Reconciliation should be successful. Additional info: --------------- Same results if enabling the featuregate "enableCommonBootImageImport".
The cause is probably the same as in Bug 2037290.
The bug that this one depends on was fixed.
Verification with a cluster running CNV 4.10.0-595 (the bundle contains the fixed ssp version 4.10.0-39) -------------------------------------------------------------------------------------------------------- * With featuregate disabled - adding/modifying a custom entry in HCO CR - propagated to SSP CR as expected, no reconciliation loops in ssp-operator pod logs. * Enabling the featuregate - SSP CR got the hard-coded templates. * Executed several automation tests (still under development). [cnv-qe-jenkins@c01-issac-410-hhrz5-executor ~]$ oc get dic -A NAMESPACE NAME AGE openshift-virtualization-os-images centos-stream8-image-cron 21s openshift-virtualization-os-images centos-stream9-image-cron 21s openshift-virtualization-os-images fedora-image-cron 21s openshift-virtualization-os-images my-image-cron 3m18s openshift-virtualization-os-images rhel8-image-cron 21s openshift-virtualization-os-images rhel9-image-cron 21s [cnv-qe-jenkins@c01-issac-410-hhrz5-executor ~]$ oc get is -n openshift-virtualization-os-images NAME IMAGE REPOSITORY TAGS UPDATED rhel8-guest image-registry.openshift-image-registry.svc:5000/openshift-virtualization-os-images/rhel8-guest latest 2 minutes ago rhel9-guest image-registry.openshift-image-registry.svc:5000/openshift-virtualization-os-images/rhel9-guest latest 2 minutes ago [cnv-qe-jenkins@c01-issac-410-hhrz5-executor ~]$ Moving to VERIFIED.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Virtualization 4.10.0 Images security and bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0947