Description of the problem: After an ACM upgrade from 2.4.3 to 2.4.4 policies are no longer copied into the cluster namespace. 

The affected policies are created by one of our operators and prior to the ACM upgrade worked correctly. Specifically, when the controller creates the policy, placementrule, and placementbinding the child policy is created in the cluster namespace. After the upgrade the child policy never shows up in the cluster namespace. After doing some digging we found error messages like this in the multicluster-operators-application-7c44cfbd5d-4vpsw multicluster-operators-placementrule container:

E0510 19:55:55.956629       1 placementrule_controller.go:226] Status update -.ztp-install/cnfde21-common-cnfde21-config-policy with err:placementrules.apps.open-cluster-management.io "cnfde21-common-cnfde21-config-policy" is forbidden: User "system:serviceaccount:openshift-cluster-group-upgrades:cluster-group-upgrades-controller-manager" cannot update resource "placementrules/status" in API group "apps.open-cluster-management.io" in the namespace "ztp-install"

The referenced ServiceAccount is bound to a Role which provides access to the necessary resources. Our operator creates this SA and role with (among others):
- apiGroups:
  - apps.open-cluster-management.io
  resources:
  - placementrules
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
When I add in placementrules/status the errors in the multicluster-operators-application are resolved and the child policies are immediately created as expected.

This issue appears to happen only after an upgrade of ACM. We saw it first (from recollection, we did not get to root cause on the first occurrence) on a hub cluster when ACM upgraded from 2.4.2 to 2.4.3, and then this week on a hub which upgraded from 2.4.3 to 2.4.4.


Release version: 2.4.4

Operator snapshot version:

OCP version: 4.9.21

Browser Info:

Steps to reproduce:
1.
2.
3.

Actual results:

Expected results:

Additional info: