Bug 2083076
| Summary: | $labels.instance is empty in the message when firing FluentdNodeDown alert | ||||||
|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Daein Park <dapark> | ||||
| Component: | Logging | Assignee: | Jeff Cantrill <jcantril> | ||||
| Status: | CLOSED DEFERRED | QA Contact: | Anping Li <anli> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 4.9 | CC: | amuller, anpicker, erooth, gvanloo, juzhao | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | logging-core | ||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2022-05-10 19:34:39 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
change the component to Logging since it's Logging alert rule Hello, what version of cluster-logging is this occurring with? Please note that only bugs against 5.0+ should be logged in JIRA. > Hello, what version of cluster-logging is this occurring with? Please note that only bugs against 5.0+ should be logged in JIRA.
Thank you for pointing it. The Cluster logging version is v5.4 as follows.
$ oc get csv
NAME DISPLAY VERSION REPLACES PHASE
cluster-logging.5.4.0-138 Red Hat OpenShift Logging 5.4.0-138 cluster-logging.5.3.5-20 Succeeded
elasticsearch-operator.5.4.0-152 OpenShift Elasticsearch Operator 5.4.0-152 elasticsearch-operator.5.3.5-20 Succeeded
logging 5.x is tracked in JIRA. Closing in favor of https://issues.redhat.com/browse/LOG-2605 |
Created attachment 1877998 [details] Observe dashboard capture file which is missing instance label in the alert messages. Description of problem: When firing FluentdNodeDown alert, the alert message is missing "$labels.instance" as follows. As a result, it does not honor "$labels.instance" label in the alert message. Check the image capture file together please. Displayed messages: ~~~ Prometheus could not scrape fluentd for more than 10m. ~~~ Alert rule definition: ~~~ - name: logging_fluentd.alerts rules: - alert: FluentdNodeDown annotations: message: Prometheus could not scrape fluentd {{ $labels.instance }} for more <--- HERE than 10m. summary: Fluentd cannot be scraped expr: absent(up{job="collector",namespace="openshift-logging"} == 1) for: 10m labels: namespace: openshift-logging service: fluentd severity: critical ~~~ Is it expected result ? How to display the "$labels.instance" label in the alert message ? Version-Release number of selected component (if applicable): $ oc version Client Version: 4.9.23 Server Version: 4.9.23 Kubernetes Version: v1.22.3+b93fd35 How reproducible: You can always reproduce this when firing the "FluentdNodeDown". Steps to Reproduce: 1. 2. 3. Actual results: Missing the "$labels.instance" value in the alert messages, even though the label is configured in the alert rules. Expected results: The "$labels.instance" value should be displayed as the alert rule is defined. Additional info: