Description of the problem: Observability Deployed - only the local cluster shows up in grafana. observability-addon is not visibile / deployed to managed cluster ``` 2022-06-21T15:57:36.857Z ERROR controller_placementrule Failed to create manifestwork {“error”: “admission webhook \“manifestworkvalidators.admission.work.open-cluster-management.io\” denied the request: the size of manifests is 52739 bytes which exceeds the 50k limit”} ``` This affects ALL managed clusters in this env (which uses RHV and bare metal clusters) Obersvability is thus not working at all, hence sev=high Release version: RHACM 2.5.0 Operator snapshot version: egistry.redhat.io/rhacm2/multicluster-observability-rhel8-operator@sha256:dd888f81c33bd51608b54fef35bd63de48ea0eb60502be8c497e1a63482b9a57 OCP version: 4.9.36 Browser Info: Chrome Additional info: hub and managed cluster are on Red Hat VPN - contact me on slack or gchat to get access and see yourself. observability operator log snippet. 2022-06-21T17:54:04.705Z INFO controller_placementrule Monitoring operator should be installed in cluster {"cluster_name": "stormshift-ocp10", "request.name": "stormshift-ocp10", "request.namespace": ""} 2022-06-21T17:54:04.705Z INFO controller_placementrule observabilityaddon already existed/unchanged {"namespace": "stormshift-ocp10"} 2022-06-21T17:54:04.705Z INFO controller_placementrule clusterrolebinding endpoint-observability-mco-rolebinding already existed/unchanged {"namespace": "stormshift-ocp10"} 2022-06-21T17:54:04.705Z INFO controller_placementrule rolebinding endpoint-observability-res-rolebinding already existed/unchanged {"namespace": "stormshift-ocp10"} 2022-06-21T17:54:04.713Z INFO controller_placementrule Creating manifestwork {"namespace": "stormshift-ocp10", "name": "stormshift-ocp10-observability"} 2022-06-21T17:54:04.745Z ERROR controller_placementrule Failed to create manifestwork {"namespace": "stormshift-ocp10", "name": "stormshift-ocp10-observability", "error": "admission webhook \"manifestworkvalidators.admission.work.open-cluster-management.io\" denied the request: the size of manifests is 52750 bytes which exceeds the 50k limit"} github.com/stolostron/multicluster-observability-operator/operators/multiclusterobservability/controllers/placementrule.createManagedClusterRes /remote-source/app/operators/multiclusterobservability/controllers/placementrule/placementrule_controller.go:431 github.com/stolostron/multicluster-observability-operator/operators/multiclusterobservability/controllers/placementrule.createAllRelatedRes /remote-source/app/operators/multiclusterobservability/controllers/placementrule/placementrule_controller.go:351 github.com/stolostron/multicluster-observability-operator/operators/multiclusterobservability/controllers/placementrule.(*PlacementRuleReconciler).Reconcile /remote-source/app/operators/multiclusterobservability/controllers/placementrule/placementrule_controller.go:158 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:114 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:311
Thx for the workaround. Step #1 did it already for me! Regarding Step #2, I did add the annotation to the config map. is that correct?
@dfroehli any update?