Bug 2227785
| Summary: | KubevirtHyperconvergedClusterOperatorCRModification alert in firing state during cnv upgrade (4.12.4->4.12.5) | ||
|---|---|---|---|
| Product: | Container Native Virtualization (CNV) | Reporter: | Ahmad <ahafe> |
| Component: | Installation | Assignee: | Simone Tiraboschi <stirabos> |
| Status: | ASSIGNED --- | QA Contact: | Natalie Gavrielov <ngavrilo> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 4.12.5 | CC: | stirabos |
| Target Milestone: | --- | ||
| Target Release: | 4.14.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | Bug | |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Description of problem: During cnv upgrade v4.12.4 to 4.12.5-50 (OCP: 4.12.26), noticed KubevirtHyperconvergedClusterOperatorCRModification alert in firing state. Version-Release number of selected component (if applicable): cnv: 4.12.4 to 4.12.5-50 (OCP: 4.12.26) How reproducible: 1 out of 1 attempt Steps to Reproduce: 1. Check alerts fired during cnv upgrade 2. 3. Actual results: {'labels': {'alertname': 'KubevirtHyperconvergedClusterOperatorCRModification', 'component_name': 'role/hyperconverged-cluster-operator-metrics', 'kubernetes_operator_component': 'hyperconverged-cluster-operator', 'kubernetes_operator_part_of': 'kubevirt', 'severity': 'warning'}, 'annotations': {'description': 'Out-of-band modification for role/hyperconverged-cluster-operator-metrics.', 'runbook_url': 'https://kubevirt.io/monitoring/runbooks/KubevirtHyperconvergedClusterOperatorCRModification', 'summary': '1 out-of-band CR modifications were detected in the last 10 minutes.'}, 'state': 'firing', 'activeAt': '2023-07-26T20:47:55.920372176Z', 'value': '1e+00'}, {'labels': {'alertname': 'KubevirtHyperconvergedClusterOperatorCRModification', 'component_name': 'servicemonitor/kubevirt-hyperconverged-operator-metrics', 'kubernetes_operator_component': 'hyperconverged-cluster-operator', 'kubernetes_operator_part_of': 'kubevirt', 'severity': 'warning'}, 'annotations': {'description': 'Out-of-band modification for servicemonitor/kubevirt-hyperconverged-operator-metrics.', 'runbook_url': 'https://kubevirt.io/monitoring/runbooks/KubevirtHyperconvergedClusterOperatorCRModification', 'summary': '1 out-of-band CR modifications were detected in the last 10 minutes.'}, 'state': 'firing', 'activeAt': '2023-07-26T20:48:55.920372176Z', 'value': '1e+00'}, {'labels': {'alertname': 'KubevirtHyperconvergedClusterOperatorCRModification', 'component_name': 'rolebinding/hyperconverged-cluster-operator-metrics', 'kubernetes_operator_component': 'hyperconverged-cluster-operator', 'kubernetes_operator_part_of': 'kubevirt', 'severity': 'warning'}, 'annotations': {'description': 'Out-of-band modification for rolebinding/hyperconverged-cluster-operator-metrics.', 'runbook_url': 'https://kubevirt.io/monitoring/runbooks/KubevirtHyperconvergedClusterOperatorCRModification', 'summary': '1 out-of-band CR modifications were detected in the last 10 minutes.'}, 'state': 'firing', 'activeAt': '2023-07-26T20:48:55.920372176Z', 'value': '1e+00'}, {'labels': {'alertname': 'KubevirtHyperconvergedClusterOperatorCRModification', 'component_name': 'service/kubevirt-hyperconverged-operator-metrics', 'kubernetes_operator_component': 'hyperconverged-cluster-operator', 'kubernetes_operator_part_of': 'kubevirt', 'severity': 'warning'}, 'annotations': {'description': 'Out-of-band modification for service/kubevirt-hyperconverged-operator-metrics.', 'runbook_url': 'https://kubevirt.io/monitoring/runbooks/KubevirtHyperconvergedClusterOperatorCRModification', 'summary': '1 out-of-band CR modifications were detected in the last 10 minutes.'}, 'state': 'firing', 'activeAt': '2023-07-26T20:48:55.920372176Z', 'value': '1e+00'}] assert not [{'activeAt': '2023-07-26T20:49:22.832759694Z', 'annotations': {'runbook_url': 'https://kubevirt.io/monitoring/runbook...omponent': 'hyperconverged-cluster-operator', 'kubernetes_operator_part_of': 'kubevirt', ...}, 'state': 'firing', ...}] Expected results: we are trying to capture all the alerts that are fired during upgrades and reduce the noise generated. Additional info: http://pastebin.test.redhat.com/1106300 must-gather.log http://pastebin.test.redhat.com/1106301