This bug was initially created as a copy of Bug #1934071 I am copying this bug because: Description of problem: Customer upgraded cluster from 4.5 to 4.6.17 After the upgrade they started to see the following behavior on the cluster: - High cpu/memory usage in master nodes. - etcd is unstable. It gets disconnected from endpoints and it has frequent leader elections. - master nodes are presenting very high cpu and memory usage. kube-apiserver and etcd are using all the resources. - api is not available unless we restart the platform, after restart api is available just for a minute or two. - When we lose access to api we are not able to log to master nodes either because cpu/memory is too high. - After some time we are able to login again to nodes, but again we lose connection with them because cpu/memory usage. - etcd was defrag moving from 3GB to 957 MB but after 10m the database size is 2GB again... - All master nodes are affected with some issue that cause etcd and api-server consume 100% memory and 100% cpu. - Hosts stay inaccessible even trough ssh. Restarting the nodes makes api-server return but it last just minutes. - Cannot send commands to nodes, crioctl and event top are not able to respond In the muust-gather there are a lots of errors from OLM: Starting: 06:34 - Ending 07:07 ============================= 2021-02-26T06:43:03.556442096-03:00 {"level":"error","ts":1614332583.5563064,"logger":"controllers.adoption","msg":"component not found","error":"InstallPlan.operators.coreos.com \"install-gssr9\" not found","stacktrace":"github.com/go- logr/zapr.(*zapLogger).Error\n\t/build/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-lifecycle-manager/pkg/controller/operators.(*AdoptionReconciler).adopt\n\t/build/pkg/controller/operators/adoption _controller.go:269\ngithub.com/operator-framework/operator-lifecycle-manager/pkg/controller/operators.(*AdoptionReconciler).adoptInstallPlan\n\t/build/pkg/controller/operators/adoption_controller.go:383\ngithub.com/operator-framework/op erator-lifecycle-manager/pkg/controller/operators.(*AdoptionReconciler).ReconcileSubscription\n\t/build/pkg/controller/operators/adoption_controller.go:169\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/build/vendor/si gs.k8s.io/controller-runtime/pkg/reconcile/reconcile.go:93\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/build/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:256\ nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/build/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller. (*Controller).worker\n\t/build/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155\nk8s.io/ apimachinery/pkg/util/wait.BackoffUntil\n\t/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pk g/util/wait.Until\n\t/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90"} ============================ wc -l = 12519 * Before the upgrade the cluster was running fine, no issues. Version-Release number of selected component (if applicable): OCP 4.6.17 How reproducible: Customer specific at this moment, I wasn't able to reproduce the issue. Additional info:
Cluster Version: 4.7.0-0.nightly-2021-04-30-074750 OLM version: 0.17.0 git commit: 48aea63e3aa0d42c6d69215b23f6132188e2220f 1) install etcd operator 2) delete csv to trigger ResolutionFailed event. 3) check catalog-operator log selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd E0430 17:16:11.877135 1 queueinformer_operator.go:290] sync {"update" "test-operators/etcd"} failed: Operation cannot be fulfilled on subscriptions.operators.coreos.com "etcd": the object has been modified; please apply your changes to the latest version and try again time="2021-04-30T17:16:11Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:11Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:11Z" level=warning msg="an error was encountered during reconciliation" error="Operation cannot be fulfilled on subscriptions.operators.coreos.com \"etcd\": the object has been modified; please apply your changes to the latest version and try again" event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd E0430 17:16:11.889078 1 queueinformer_operator.go:290] sync {"update" "test-operators/etcd"} failed: Operation cannot be fulfilled on subscriptions.operators.coreos.com "etcd": the object has been modified; please apply your changes to the latest version and try again time="2021-04-30T17:16:11Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:11Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:12Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:12Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:12Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:12Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:12Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:12Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:12Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:12Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:12Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:13Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{redhat-marketplace openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="failed to construct operator from bundle, continuing: couldn't parse bundle path, missing provided and required apis" catalog="{community-operators openshift-marketplace}" time="2021-04-30T17:16:14Z" level=warning msg="no installplan found with matching generation, creating new one" id=VH0dz namespace=test-operators time="2021-04-30T17:16:14Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:14Z" level=info msg=syncing id=vnvkS ip=install-m4q88 namespace=test-operators phase= time="2021-04-30T17:16:14Z" level=info msg="skip processing installplan without status - subscription sync responsible for initial status" id=vnvkS ip=install-m4q88 namespace=test-operators phase= time="2021-04-30T17:16:14Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:14Z" level=info msg=syncing id=tLEhs ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:14Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:15Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:15Z" level=warning msg="an error was encountered during reconciliation" error="Operation cannot be fulfilled on subscriptions.operators.coreos.com \"etcd\": the object has been modified; please apply your changes to the latest version and try again" event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd E0430 17:16:15.044118 1 queueinformer_operator.go:290] sync {"update" "test-operators/etcd"} failed: Operation cannot be fulfilled on subscriptions.operators.coreos.com "etcd": the object has been modified; please apply your changes to the latest version and try again time="2021-04-30T17:16:15Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:15Z" level=warning msg="status not equal, updating..." id=tLEhs ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:15Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:15Z" level=warning msg="an error was encountered during reconciliation" error="Operation cannot be fulfilled on subscriptions.operators.coreos.com \"etcd\": the object has been modified; please apply your changes to the latest version and try again" event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd E0430 17:16:15.359552 1 queueinformer_operator.go:290] sync {"update" "test-operators/etcd"} failed: Operation cannot be fulfilled on subscriptions.operators.coreos.com "etcd": the object has been modified; please apply your changes to the latest version and try again time="2021-04-30T17:16:15Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd E0430 17:16:15.559014 1 queueinformer_operator.go:290] sync {"update" "test-operators/install-m4q88"} failed: failed to update installplan bundle lookups: Operation cannot be fulfilled on installplans.operators.coreos.com "install-m4q88": the object has been modified; please apply your changes to the latest version and try again time="2021-04-30T17:16:15Z" level=info msg=syncing id=bznvQ ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:15Z" level=warning msg="status not equal, updating..." id=bznvQ ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:15Z" level=info msg=syncing id=yw5M+ ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:15Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:15Z" level=warning msg="status not equal, updating..." id=yw5M+ ip=install-m4q88 namespace=test-operators phase=Installing E0430 17:16:16.361063 1 queueinformer_operator.go:290] sync {"update" "test-operators/install-m4q88"} failed: failed to update installplan bundle lookups: Operation cannot be fulfilled on installplans.operators.coreos.com "install-m4q88": the object has been modified; please apply your changes to the latest version and try again time="2021-04-30T17:16:16Z" level=info msg=syncing id=7e7vp ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:16Z" level=info msg=syncing id=0bzMB ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:21Z" level=info msg=syncing id=gIi9f ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:26Z" level=info msg=syncing id=aq8L2 ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:31Z" level=info msg=syncing id=EcQSw ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:36Z" level=info msg=syncing id=N73eA ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:36Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/c8ec2a554f508a32921efcee390f67dd68efd5c9dd0ea8e76d8795d897e8cd2 key=etcdrestores.etcd.database.coreos.com.crd.yaml time="2021-04-30T17:16:36Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/c8ec2a554f508a32921efcee390f67dd68efd5c9dd0ea8e76d8795d897e8cd2 key=etcdbackups.etcd.database.coreos.com.crd.yaml time="2021-04-30T17:16:36Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/c8ec2a554f508a32921efcee390f67dd68efd5c9dd0ea8e76d8795d897e8cd2 key=etcdclusters.etcd.database.coreos.com.crd.yaml time="2021-04-30T17:16:36Z" level=info msg="added to bundle, Kind=ClusterServiceVersion" configmap=openshift-marketplace/c8ec2a554f508a32921efcee390f67dd68efd5c9dd0ea8e76d8795d897e8cd2 key=etcdoperator.v0.9.4.clusterserviceversion.yaml time="2021-04-30T17:16:36Z" level=warning msg="status not equal, updating..." id=N73eA ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:36Z" level=info msg=syncing id=j10fs ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:36Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:36Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/c8ec2a554f508a32921efcee390f67dd68efd5c9dd0ea8e76d8795d897e8cd2 key=etcdrestores.etcd.database.coreos.com.crd.yaml time="2021-04-30T17:16:36Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/c8ec2a554f508a32921efcee390f67dd68efd5c9dd0ea8e76d8795d897e8cd2 key=etcdbackups.etcd.database.coreos.com.crd.yaml time="2021-04-30T17:16:36Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/c8ec2a554f508a32921efcee390f67dd68efd5c9dd0ea8e76d8795d897e8cd2 key=etcdclusters.etcd.database.coreos.com.crd.yaml time="2021-04-30T17:16:36Z" level=info msg="added to bundle, Kind=ClusterServiceVersion" configmap=openshift-marketplace/c8ec2a554f508a32921efcee390f67dd68efd5c9dd0ea8e76d8795d897e8cd2 key=etcdoperator.v0.9.4.clusterserviceversion.yaml W0430 17:16:36.435400 1 warnings.go:70] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0430 17:16:36.454040 1 warnings.go:70] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0430 17:16:36.466276 1 warnings.go:70] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition time="2021-04-30T17:16:36Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:36Z" level=info msg=syncing id=xDruL ip=install-m4q88 namespace=test-operators phase=Installing time="2021-04-30T17:16:36Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:36Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/c8ec2a554f508a32921efcee390f67dd68efd5c9dd0ea8e76d8795d897e8cd2 key=etcdrestores.etcd.database.coreos.com.crd.yaml time="2021-04-30T17:16:36Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/c8ec2a554f508a32921efcee390f67dd68efd5c9dd0ea8e76d8795d897e8cd2 key=etcdbackups.etcd.database.coreos.com.crd.yaml time="2021-04-30T17:16:36Z" level=info msg="added to bundle, Kind=CustomResourceDefinition" configmap=openshift-marketplace/c8ec2a554f508a32921efcee390f67dd68efd5c9dd0ea8e76d8795d897e8cd2 key=etcdclusters.etcd.database.coreos.com.crd.yaml time="2021-04-30T17:16:36Z" level=info msg="added to bundle, Kind=ClusterServiceVersion" configmap=openshift-marketplace/c8ec2a554f508a32921efcee390f67dd68efd5c9dd0ea8e76d8795d897e8cd2 key=etcdoperator.v0.9.4.clusterserviceversion.yaml W0430 17:16:36.805906 1 warnings.go:70] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0430 17:16:36.810279 1 warnings.go:70] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition W0430 17:16:36.814804 1 warnings.go:70] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition time="2021-04-30T17:16:37Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:37Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:37Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:37Z" level=info msg=syncing id=GuVz8 ip=install-m4q88 namespace=test-operators phase=Complete time="2021-04-30T17:16:37Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:37Z" level=warning msg="an error was encountered during reconciliation" error="Operation cannot be fulfilled on subscriptions.operators.coreos.com \"etcd\": the object has been modified; please apply your changes to the latest version and try again" event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd E0430 17:16:37.799404 1 queueinformer_operator.go:290] sync {"update" "test-operators/etcd"} failed: Operation cannot be fulfilled on subscriptions.operators.coreos.com "etcd": the object has been modified; please apply your changes to the latest version and try again time="2021-04-30T17:16:37Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:38Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:16:41Z" level=info msg=syncing id=K5rfB ip=install-m4q88 namespace=test-operators phase=Complete time="2021-04-30T17:16:41Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:18:37Z" level=info msg="catalog update required at 2021-04-30 17:18:37.47048493 +0000 UTC m=+13042.199636896" CatalogSource=community-operators time="2021-04-30T17:18:50Z" level=info msg="catalog polling result: no update" CatalogSource=community-operators time="2021-04-30T17:20:33Z" level=info msg="Adding related objects for operator-lifecycle-manager-catalog" time="2021-04-30T17:24:01Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:24:01Z" level=info msg=syncing id=RRx6Q ip=install-m4q88 namespace=test-operators phase=Complete time="2021-04-30T17:24:01Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:24:01Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:24:01Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=/apis/operators.coreos.com/v1alpha1/namespaces/test-operators/subscriptions/etcd time="2021-04-30T17:24:02Z" level=info msg="catalog update required at 2021-04-30 17:24:02.080403028 +0000 UTC m=+13366.809554997" CatalogSource=redhat-operators time="2021-04-30T17:24:02Z" level=info msg="catalog update required at 2021-04-30 17:24:02.081847959 +0000 UTC m=+13366.810999925" CatalogSource=redhat-marketplace time="2021-04-30T17:24:13Z" level=info msg="catalog polling result: no update" CatalogSource=redhat-marketplace time="2021-04-30T17:24:16Z" level=info msg="catalog polling result: no update" CatalogSource=redhat-operators time="2021-04-30T17:24:23Z" level=info msg="catalog update required at 2021-04-30 17:24:23.88742903 +0000 UTC m=+13388.616580995" CatalogSource=certified-operators time="2021-04-30T17:24:42Z" level=info msg="catalog polling result: no update" CatalogSource=certified-operators time="2021-04-30T17:25:38Z" level=info msg="Adding related objects for operator-lifecycle-manager-catalog" There is no such log "creating installplan" after the "ResolutionFailed" event happens. LGTM, verified
This bug will be shipped as part of next z-stream release 4.7.11 on May 19th, as 4.7.10 was dropped due to a blocker https://bugzilla.redhat.com/show_bug.cgi?id=1958518.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.7.11 bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:1550