Bug 2291298
| Summary: | PrometheusRuleFailures alert with rule_group /etc/prometheus/rules/prometheus-k8s-rulefiles-0/openshift-storage-prometheus-ceph-rules-* after change of in-transit encryption configuration | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat OpenShift Data Foundation | Reporter: | Filip Balák <fbalak> |
| Component: | ceph-monitoring | Assignee: | arun kumar mohan <amohan> |
| Status: | CLOSED NOTABUG | QA Contact: | Harish NV Rao <hnallurv> |
| Severity: | low | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 4.16 | CC: | amohan, nthomas, odf-bz-bot |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2024-08-30 11:25:31 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Description of problem (please be detailed as possible and provide log snippests): After following steps are executed on cluster with set in-transit encryption: 1. Enable in-transit Encryption if not Enabled. 2. Verify in-transit Encryption is Enable on setup. 3. Disable Encryption 4. Verify in-transit encryption configuration is removed. 5. Enable encryption Again and verify it. 6. Verify in-transit encryption config is exists. There is a similar alert for some time: {'labels': {'alertname': 'PrometheusRuleFailures', 'container': 'kube-rbac-proxy', 'endpoint': 'metrics', 'instance': '10.131.0.31:9092', 'job': 'prometheus-k8s', 'namespace': 'openshift-monitoring', 'pod': 'prometheus-k8s-0', 'rule_group': '/etc/prometheus/rules/prometheus-k8s-rulefiles-0/openshift-storage-prometheus-ceph-rules-737a4da2-7bec-4c21-a113-6e3b8ac91d83.yaml;telemeter.rules', 'service': 'prometheus-k8s', 'severity': 'warning'}, 'annotations': {'description': 'Prometheus openshift-monitoring/prometheus-k8s-0 has failed to evaluate 4 rules in the last 5m.', 'runbook_url': 'https://github.com/openshift/runbooks/blob/master/alerts/cluster-monitoring-operator/PrometheusRuleFailures.md', 'summary': 'Prometheus is failing rule evaluations.'}, 'state': 'pending', 'activeAt': '2024-06-01T02:34:11.604846708Z', 'value': '4.130400000000001e+00'}, Version of all relevant components (if applicable): ODF 4.16.0-22 Can this issue reproducible? yes Steps to Reproduce: 1. Enable in-transit Encryption if not Enabled. 2. Verify in-transit Encryption is Enable on setup. 3. Disable Encryption 4. Verify in-transit encryption configuration is removed. 5. Enable encryption Again and verify it. 6. Verify in-transit encryption config is exists. 7. Check alerts Actual results: There is an alert PrometheusRuleFailures Expected results: There is no PrometheusRuleFailures alert Additional info: This is automated by test case test_intransit_encryption_enable_disable_statetransition (https://github.com/red-hat-storage/ocs-ci/blob/0cf981385c50d65b29201505965b6e31faf672a5/tests/functional/encryption/test_intransit_encryption_sanity.py#L38)