How reproducible: Always Description of problem: NodeHasIntegrityFailure alert doesn't set namespace label after upgrade file-integrity-operator from v0.1.24>v0.1.30. Version-Release number of selected component (if applicable): File-integrity-operator.v0.1.30 How reproducible: always Steps to Reproduce: 1. install File Integrity Operator v0.1.24 2. Create fileintegrity and make NodeHasIntegrityFailure fire from a node 3. Upgrade to File-integrity-operator.v0.1.30 and inspect the alert lables Actual results: NodeHasIntegrityFailure doesn't set namespace label Expected results: NodeHasIntegrityFailure should be fired from the namespace the operator is is installed in (by default openshift-file-integrity) Additional info: There is no such issue if for fresh install File-integrity-operator.v0.1.30
The monitoring Rule isn't updated if it has been created by the operator previously: https://github.com/openshift/file-integrity-operator/blob/master/cmd/manager/operator.go#L295 So in this situation, the namespace addition from 0.1.27 is not applied. Deleting the Rule and restarting the operator pod would be a workaround.
verification pass with file-integrity-operator.v0.1.31 + 4.12.0-0.nightly-2022-10-20-104328 Tried to verify with below two scenarios: 1. install File Integrity Operator v0.1.24 2. Create fileintegrity and make NodeHasIntegrityFailure fire from a node 3. Upgrade to File-integrity-operator.v0.1.31, retrigger a NodeHasIntegrityFailure, and inspect the alert lables. NodeHasIntegrityFailure has namespace label. $ oc get csv NAME DISPLAY VERSION REPLACES PHASE file-integrity-operator.v0.1.31 File Integrity Operator 0.1.31 file-integrity-operator.v0.1.30 Succeeded $ ALERT_MANAGER=$(oc get route alertmanager-main -n openshift-monitoring -o jsonpath='{@.spec.host}') $ curl -k -H "Authorization: Bearer $(oc create token prometheus-k8s -n openshift-monitoring)" https://$ALERT_MANAGER/api/v1/alerts | jq -r { "status": "success", "data": [ ... { "labels": { "alertname": "NodeHasIntegrityFailure", "namespace": "openshift-file-integrity", "node": "xiyuan21-1-6xq7z-master-1.c.openshift-qe.internal", "openshift_io_alert_source": "platform", "prometheus": "openshift-monitoring/k8s", "severity": "warning" }, "annotations": { "description": "Node xiyuan21-1-6xq7z-master-1.c.openshift-qe.internal has an integrity check status of Failed for more than 1 second.", "summary": "Node xiyuan21-1-6xq7z-master-1.c.openshift-qe.internal has a file integrity failure" }, "startsAt": "2022-10-21T09:14:27.768Z", "endsAt": "2022-10-21T11:24:27.768Z", "generatorURL": "https:/console-openshift-console.apps.xiyuan21-1.qe.gcp.devcluster.openshift.com/monitoring/graph?g0.expr=file_integrity_operator_node_failed%7Bnode%3D~%22.%2B%22%7D+%2A+on+%28node%29+kube_node_info+%3E+0&g0.tab=1", "status": { "state": "active", "silencedBy": null, "inhibitedBy": null }, "receivers": [ "Default" ], "fingerprint": "2c4439dd0333d79d" }, ...
Per https://bugzilla.redhat.com/show_bug.cgi?id=2112394#c3 and https://bugzilla.redhat.com/show_bug.cgi?id=2112394#c7, move it to verified.
Update one typo error in https://bugzilla.redhat.com/show_bug.cgi?id=2112394#c7 in step1: should be: 1. install File Integrity Operator v0.1.30
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift File Integrity Operator bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:7095