Bug 2083039
| Summary: | Got a lot of "Reconciler error" in cdi-deployment log after adding custom DataImportCron to hco | |||
|---|---|---|---|---|
| Product: | Container Native Virtualization (CNV) | Reporter: | Yan Du <yadu> | |
| Component: | Storage | Assignee: | Arnon Gilboa <agilboa> | |
| Status: | CLOSED ERRATA | QA Contact: | Yan Du <yadu> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 4.10.1 | CC: | agilboa, cnv-qe-bugs, mrashish, ngavrilo | |
| Target Milestone: | --- | |||
| Target Release: | 4.10.3 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | CNV v4.10.2-12 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2091982 (view as bug list) | Environment: | ||
| Last Closed: | 2022-07-20 16:01:13 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 2054782 | |||
| Bug Blocks: | 2091982 | |||
Created attachment 1877988 [details]
cdi deployment log
Oops, picked wrong version, corrected Test on CNV-v4.10.3-4 The "Reconciler error" in cdi-deployment log has been reduced Got 3 times "Reconciler error" now:
{"level":"error","ts":1656315188.8290644,"logger":"controller.dataimportcron-controller","msg":"Reconciler error","name":"data-import-test1","namespace":"openshift-virtualization-os-images","error":"Operation cannot be fulfilled on datasources.cdi.kubevirt.io \"data-source-test1\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1656315189.053071,"logger":"controller.dataimportcron-controller","msg":"Reconciler error","name":"data-import-test1","namespace":"openshift-virtualization-os-images","error":"Operation cannot be fulfilled on dataimportcrons.cdi.kubevirt.io \"data-import-test1\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1656315198.178084,"logger":"controller.datavolume-controller","msg":"Reconciler error","name":"data-source-test1-3b3fc310abea","namespace":"openshift-virtualization-os-images","error":"Operation cannot be fulfilled on datavolumes.cdi.kubevirt.io \"data-source-test1-3b3fc310abea\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Virtualization 4.10.3 Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2022:5675 |
Description of problem: Got a lot of "Reconciler error" in cdi-deployment log after adding custom DataImportCron to hco Version-Release number of selected component (if applicable): CNV-v4.10.1-101 How reproducible: Always Steps to Reproduce: 1. Add custom DataImportCron to HCO eg: --------------8<------------------ spec: certConfig: ca: duration: 48h0m0s renewBefore: 24h0m0s server: duration: 24h0m0s renewBefore: 12h0m0s dataImportCronTemplates: - metadata: annotations: cdi.kubevirt.io/storage.bind.immediate.requested: "true" name: data-import-test2 spec: managedDataSource: data-source-test2 retentionPolicy: None schedule: '* * * * *' template: spec: source: registry: pullMethod: node url: docker://quay.io/containerdisks/fedora:35 storage: resources: requests: storage: 10Gi featureGates: enableCommonBootImageImport: true --------------8<------------------ Actual results: Got below error frequently till the import finished, log file is attached. $ oc logs cdi-deployment-7bbb74d84d-hjkjs -n openshift-cnv | grep Reconciler | grep data-source-test2 {"level":"error","ts":1652077998.9596689,"logger":"controller.dataimportcron-controller","msg":"Reconciler error","name":"data-source-test2","namespace":"openshift-virtualization-os-images","error":"Operation cannot be fulfilled on datasources.cdi.kubevirt.io \"data-source-test2\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"} {"level":"error","ts":1652078009.948743,"logger":"controller.datavolume-controller","msg":"Unable to update datavolume","name":"data-source-test2-3b3fc310abea","error":"Operation cannot be fulfilled on datavolumes.cdi.kubevirt.io \"data-source-test2-3b3fc310abea\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"kubevirt.io/containerized-data-importer/pkg/controller.(*DatavolumeReconciler).reconcileDataVolumeStatus\n\t/remote-source/app/pkg/controller/datavolume-controller.go:2144\nkubevirt.io/containerized-data-importer/pkg/controller.(*DatavolumeReconciler).Reconcile\n\t/remote-source/app/pkg/controller/datavolume-controller.go:526\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"} {"level":"error","ts":1652078009.9488642,"logger":"controller.datavolume-controller","msg":"Reconciler error","name":"data-source-test2-3b3fc310abea","namespace":"openshift-virtualization-os-images","error":"Operation cannot be fulfilled on datavolumes.cdi.kubevirt.io \"data-source-test2-3b3fc310abea\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"} {"level":"error","ts":1652078010.129169,"logger":"controller.import-controller","msg":"Reconciler error","name":"data-source-test-3b3fc310abea","namespace":"openshift-virtualization-os-images","error":"Operation cannot be fulfilled on persistentvolumeclaims \"data-source-test2-3b3fc310abea\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"} {"level":"error","ts":1652078010.1938965,"logger":"controller.import-controller","msg":"Reconciler error","name":"data-source-test2-3b3fc310abea","namespace":"openshift-virtualization-os-images","error":"Operation cannot be fulfilled on persistentvolumeclaims \"data-source-test2-3b3fc310abea\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"} $ oc get dv data-source-test2-3b3fc310abea NAME PHASE PROGRESS RESTARTS AGE data-source-test2-3b3fc310abea Succeeded 100.0% 18m Expected results: Better not to reduce the noise Additional info: log attached