Description of problem: The `key` and `ops-key` in the secret/curator turn to empty after running for 4h20m with forwarding logs to external logstore enabled, other secrets were not changed. The CLO was repeating 'level=info msg="Updating status of Curator"'. All the error messages in the CLO pod are in the below: $ oc logs cluster-logging-operator-58dccd9db-kvwjm |grep error time="2019-12-09T03:01:01Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:02:21Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" {"level":"error","ts":1575860714.0120184,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"logforwarding-controller","request":"openshift-logging/instance","error":"Operation cannot be fulfilled on logforwardings.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} time="2019-12-09T03:05:17Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:10:15Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" {"level":"error","ts":1575861276.8150146,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"logforwarding-controller","request":"openshift-logging/instance","error":"Operation cannot be fulfilled on logforwardings.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} time="2019-12-09T03:16:01Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:21:41Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:26:45Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:34:25Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:42:41Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:42:44Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:42:45Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:48:26Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:56:20Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:58:16Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:58:17Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T03:58:18Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:06:45Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:06:48Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:06:49Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:15:31Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:21:25Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:24:23Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:29:41Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:29:41Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:39:13Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:39:14Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:39:15Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:44:29Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:44:30Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:44:31Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:50:50Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:50:51Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:58:03Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T04:58:04Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T05:12:19Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T05:21:04Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T05:26:19Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T05:32:30Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T05:32:33Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T05:39:35Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T05:39:38Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T05:39:39Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T05:53:39Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T05:53:40Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T05:53:41Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:00:14Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:00:15Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:06:48Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:06:49Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" {"level":"error","ts":1575871655.6114109,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"logforwarding-controller","request":"openshift-logging/instance","error":"Operation cannot be fulfilled on logforwardings.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} time="2019-12-09T06:07:38Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:14:55Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:14:56Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:14:57Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:19:51Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:20:44Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" {"level":"error","ts":1575872525.808817,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"logforwarding-controller","request":"openshift-logging/instance","error":"Operation cannot be fulfilled on logforwardings.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} time="2019-12-09T06:29:02Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" {"level":"error","ts":1575872996.0121827,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"logforwarding-controller","request":"openshift-logging/instance","error":"Operation cannot be fulfilled on logforwardings.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} time="2019-12-09T06:29:59Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:35:01Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" {"level":"error","ts":1575873372.409862,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"logforwarding-controller","request":"openshift-logging/instance","error":"Operation cannot be fulfilled on logforwardings.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} time="2019-12-09T06:37:42Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:37:44Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" {"level":"error","ts":1575873501.62436,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"logforwarding-controller","request":"openshift-logging/instance","error":"Operation cannot be fulfilled on logforwardings.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} {"level":"error","ts":1575873754.6148715,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"logforwarding-controller","request":"openshift-logging/instance","error":"Operation cannot be fulfilled on logforwardings.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} time="2019-12-09T06:42:37Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:46:16Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:52:08Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:52:11Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" {"level":"error","ts":1575874591.614392,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"logforwarding-controller","request":"openshift-logging/instance","error":"Operation cannot be fulfilled on logforwardings.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} time="2019-12-09T06:56:34Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:59:28Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:59:29Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:59:32Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T06:59:33Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:03:23Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:04:38Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:06:30Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:13:10Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:13:11Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:19:02Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:19:58Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:19:59Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:22:01Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:22:04Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:22:19Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:26:47Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:27:58Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:35:24Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:35:26Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:37:09Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:45:17Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T07:45:19Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" {"level":"error","ts":1575877685.0138645,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"logforwarding-controller","request":"openshift-logging/instance","error":"Operation cannot be fulfilled on logforwardings.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} {"level":"error","ts":1575878618.6158159,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"logforwarding-controller","request":"openshift-logging/instance","error":"Operation cannot be fulfilled on logforwardings.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/openshift/cluster-logging-operator/_output/src/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} time="2019-12-09T08:03:41Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T08:10:20Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T08:18:51Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T08:18:52Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T08:24:04Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" time="2019-12-09T08:33:00Z" level=error msg="Error updating &TypeMeta{Kind:Secret,APIVersion:v1,}: Operation cannot be fulfilled on secrets \"kibana-proxy\": the object has been modified; please apply your changes to the latest version and try again" Version-Release number of selected component (if applicable): ose-cluster-logging-operator-v4.4.0-201912080435 How reproducible: Not sure, only saw this issue twice. Steps to Reproduce: 1.deploy clusterlogging 2.enable logforwarding to forward logs to external elasticsearch as unsecure 3.several hours later, check the curator status, the curator pod was in Error status Actual results: Expected results: Additional info:
Please retest with the lastest 4.4 images. There has been significant work to managing certificates in the operator which may have resolve this.
I'm not able to reproduce this issue with ose-cluster-logging-operator-v4.4.0-202002022216, move this bug to VERIFIED.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0581