Wait the success rates of this https://github.com/openshift/origin/pull/26282 backport PR is merged.
After given PR has merged. This problem seems to be solved according to the results of; https://search.ci.openshift.org/?search=Prometheus+when+installed+on+the+cluster+shouldn%27t+report+any+alerts+in+firing+state+apart+from+Watchdog+and+AlertmanagerReceiversNotConfigured&maxAge=168h&context=1&type=junit&name=metal&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job https://sippy.ci.openshift.org/sippy-ng/jobs/4.7/runs?filters=%7B%22items%22%3A%5B%7B%22columnField%22%3A%22name%22%2C%22operatorValue%22%3A%22equals%22%2C%22value%22%3A%22periodic-ci-openshift-release-master-nightly-4.7-upgrade-from-stable-4.6-e2e-metal-ipi-upgrade%22%7D%5D%7D&sortField=timestamp&sort=desc Latest e2e-metal failure due to this Prometheus test was 3 days ago.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:3759