Bug 2291298

Summary: PrometheusRuleFailures alert with rule_group /etc/prometheus/rules/prometheus-k8s-rulefiles-0/openshift-storage-prometheus-ceph-rules-* after change of in-transit encryption configuration
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Filip Balák <fbalak>
Component: ceph-monitoringAssignee: arun kumar mohan <amohan>
Status: CLOSED NOTABUG QA Contact: Harish NV Rao <hnallurv>
Severity: low Docs Contact:
Priority: unspecified    
Version: 4.16CC: amohan, nthomas, odf-bz-bot
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-08-30 11:25:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Filip Balák 2024-06-11 11:06:04 UTC
Description of problem (please be detailed as possible and provide log
snippests):
After following steps are executed on cluster with set in-transit encryption:
        1. Enable in-transit Encryption if not Enabled.
        2. Verify in-transit Encryption is Enable on setup.
        3. Disable Encryption
        4. Verify in-transit encryption configuration is removed.
        5. Enable encryption Again and verify it.
        6. Verify in-transit encryption config is exists.

There is a similar alert for some time:
{'labels': {'alertname': 'PrometheusRuleFailures', 'container': 'kube-rbac-proxy', 'endpoint': 'metrics', 'instance': '10.131.0.31:9092', 'job': 'prometheus-k8s', 'namespace': 'openshift-monitoring', 'pod': 'prometheus-k8s-0', 'rule_group': '/etc/prometheus/rules/prometheus-k8s-rulefiles-0/openshift-storage-prometheus-ceph-rules-737a4da2-7bec-4c21-a113-6e3b8ac91d83.yaml;telemeter.rules', 'service': 'prometheus-k8s', 'severity': 'warning'}, 'annotations': {'description': 'Prometheus openshift-monitoring/prometheus-k8s-0 has failed to evaluate 4 rules in the last 5m.', 'runbook_url': 'https://github.com/openshift/runbooks/blob/master/alerts/cluster-monitoring-operator/PrometheusRuleFailures.md', 'summary': 'Prometheus is failing rule evaluations.'}, 'state': 'pending', 'activeAt': '2024-06-01T02:34:11.604846708Z', 'value': '4.130400000000001e+00'},

Version of all relevant components (if applicable):
ODF 4.16.0-22

Can this issue reproducible?
yes

Steps to Reproduce:
1. Enable in-transit Encryption if not Enabled.
2. Verify in-transit Encryption is Enable on setup.
3. Disable Encryption
4. Verify in-transit encryption configuration is removed.
5. Enable encryption Again and verify it.
6. Verify in-transit encryption config is exists.
7. Check alerts

Actual results:
There is an alert PrometheusRuleFailures

Expected results:
There is no PrometheusRuleFailures alert

Additional info:
This is automated by test case test_intransit_encryption_enable_disable_statetransition (https://github.com/red-hat-storage/ocs-ci/blob/0cf981385c50d65b29201505965b6e31faf672a5/tests/functional/encryption/test_intransit_encryption_sanity.py#L38)