Description of problem: The CAO and MAO now contain tombstone manifests to ensure that deprecated resources are cleaned up during an upgrade. These manifests need to carry the annotation `include.release.openshift.io/self-managed-high-availability: "true"` to be properly processed by the CVO.
Validated and working fine. Steps : 1.Create a cluster on 4.7 , using flexi job and check prometheus rules present [miyadav@miyadav aws]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.7.0-0.nightly-2021-08-16-204701 True False 3m56s Cluster version is 4.7.0-0.nightly-2021-08-16-204701 [miyadav@miyadav aws]$ oc get prometheusrule -n openshift-machine-api NAME AGE cluster-autoscaler-operator-rules 22m machine-api-operator-prometheus-rules 37m [miyadav@miyadav aws]$ 2.edit promethues rule - cluster-autoscaler-operator rules as below , . . metadata: annotations: exclude.release.openshift.io/internal-openshift-hosted: "true" include.release.openshift.io/self-managed-high-availability: "true" include.release.openshift.io/single-node-developer: "true" release.openshift.io/delete: "true" . . [miyadav@miyadav aws]$ oc edit prometheusrule cluster-autoscaler-operator-rules -n openshift-machine-api prometheusrule.monitoring.coreos.com/machine-api-operator-prometheus-rules edited 3. Validate rules after upgraded to 4.8 NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.8.0-0.nightly-2021-08-18-111855 True False 5m27s Cluster version is 4.8.0-0.nightly-2021-08-18-111855 [miyadav@miyadav aws]$ oc edit prometheusrule cluster-autoscaler-operator-rules -n openshift-machine-api Edit cancelled, no changes made. 4.Start cluster upgrade to 4.9 , once completed check if cluster-autoscaler-operator-rules is present or not Expected and Actual [miyadav@miyadav aws]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.9.0-0.nightly-2021-08-18-033031 True False 10s Cluster version is 4.9.0-0.nightly-2021-08-18-033031 [miyadav@miyadav aws]$ oc edit prometheusrule cluster-autoscaler-operator-rules -n openshift-machine-api Error from server (NotFound): prometheusrules.monitoring.coreos.com "cluster-autoscaler-operator-rules" not found [miyadav@miyadav aws]$ Additional info: in 4.9 - [miyadav@miyadav aws]$ oc get prometheusrule -n openshift-machine-api NAME AGE machine-api-operator-prometheus-rules 3h47m [miyadav@miyadav aws]$ On the basis of these results moving to VERIFIED
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:3759