Fedora Account System
Red Hat Associate
Red Hat Customer
Description of the problem: We are writing a Kubernetes operator that creates ACM secondary resources. When the operator creates a primary resource that itself causes an ACM policy creation, the policy propagator pod logs shows this error: <snip> "level":"error","ts":1624022737.4564233,"logger":"policy-propagator","msg":"Failed to create replicated policy...","Policy-Namespace":"default","Policy-Name":"group1-batch-1-policy-deployment-policy","Namespace":"spoke1","Name":"default.group1-batch-1-policy-deployment-policy","error":"policies.policy.open-cluster-management.io \"default.group1-batch-1-policy-deployment-policy\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/remote-source/deps/gomod/pkg/mod/github.com/go-logr/zapr.1/zapr.go:128\ngithub.com/open-cluster-management/governance-policy-propagator/pkg/controller/propagator.(*ReconcilePolicy).handleDecision\n\t/remote-source/app/pkg/controller/propagator/propagation.go:213\ngithub.com/open-cluster-management/governance-policy-propagator/pkg/controller/propagator.(*ReconcilePolicy).handleRootPolicy\n\t/remote-source/app/pkg/controller/propagator/propagation.go:81\ngithub.com/open-cluster-management/governance-policy-propagator/pkg/controller/propagator.(*ReconcilePolicy).Reconcile\n\t/remote-source/app/pkg/controller/propagator/policy_propagator_controller.go:153\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.0/pkg/internal/controller/controller.go:256\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.0/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.0/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:90"} {"level":"error","ts":1624022737.4568074,"logger":"controller-runtime.controller","msg":"Reconciler error","controller":"policy-propagator","request":"default/group1-batch-1-policy-deployment-policy","error":"policies.policy.open-cluster-management.io \"default.group1-batch-1-policy-deployment-policy\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/remote-source/deps/gomod/pkg/mod/github.com/go-logr/zapr.1/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.0/pkg/internal/controller/controller.go:258\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.0/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.0/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:90"} </snip> This is due to the grc clusterrole not having the update verb on group/finalizers resource of our operator. Adding this to the clusterrole fixes the issue: - apiGroups: - '*' resources: - '*/finalizers' verbs: - update We kindly request to add these changes to the clusterrole specs, just so 3rd party operators can leverage ACM as an engine to perform changes on managed clusters. Release version: 2.2 Operator snapshot version: OCP version: 4.8 Browser Info: Steps to reproduce: 1. Clone https://github.com/rcarrillocruz/cluster-group-lcm/commit/d7b5c8ff942085fe9525f40f996da5c668b902c1 2. Run 'make deploy' with a KUBECONFIG targetting a hub cluster with ACM 3. Run oc apply -f group.yaml (group.yaml is located in the cloned repo) Actual results: Policy is not properly propagated to managed clusters and grc policy propagator pod logs shows error similar to the description above. Expected results: Policy propagator can update finalizers resources managed by 3rd party operators and policies can be propagated to managed clusters correctly. Additional info:
I was able to recreate this issue and certainly see the value of adding these roles. I'll be reviewing this change with our secure engineering and rbac experts and will work on adding this so your operator will work as expected. Thanks for trying out this scenario and reporting the bug!
G2Bsync 867070650 comment gparvin Wed, 23 Jun 2021 18:37:58 UTC G2Bsync Discussed with customer and found out this is not needed for ACM 2.2 z stream but delivery in to ACM 2.3 is ok.