Bug 1819612 - deduplication in thanos querier for rules coming from thanos ruler is not working
Summary: deduplication in thanos querier for rules coming from thanos ruler is not wor...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Monitoring
Version: 4.5
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: 4.5.0
Assignee: Lili Cosic
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-01 07:50 UTC by Junqi Zhao
Modified: 2020-07-13 17:25 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-07-13 17:24:43 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-monitoring-operator pull 729 0 None closed Bug 1819612: jsonnet/thanos-querier.jsonnet: Configure thanos querier replica-label 2020-06-23 09:44:04 UTC
Red Hat Product Errata RHBA-2020:2409 0 None None None 2020-07-13 17:25:14 UTC

Description Junqi Zhao 2020-04-01 07:50:32 UTC
Description of problem:
create PrometheusRule under user namespace, and search ALERTS{alertname="Watchdog",alertstate="firing",namespace="test3"} in thanos-querier, there is not difference if deduplication and partial response are selected or don't selected.

Steps: enable techPreviewUserWorkload and do the followings
***********************************************
# oc new-project test3
# oc create -f - << EOF
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: test3.rules
spec:
  groups:
  - name: alerting rules
    rules:
    - alert: Watchdog
      expr: vector(1)
      labels:
        severity: none
      message:
        This is an alert meant to ensure that the entire alerting pipeline is functional.
EOF
***********************************************

search in thanos-querier(deduplication and partial response are selected)
ALERTS{alertname="Watchdog",alertstate="firing",namespace="test3"}
Element 	Value
ALERTS{alertname="Watchdog",alertstate="firing",namespace="test3",severity="none",thanos_ruler_replica="thanos-ruler-user-workload-0"}	1
ALERTS{alertname="Watchdog",alertstate="firing",namespace="test3",severity="none",thanos_ruler_replica="thanos-ruler-user-workload-1"}	1

don't select deduplication and partial response, the result is the same with above
ALERTS{alertname="Watchdog",alertstate="firing",namespace="test3"}
Element 	Value
ALERTS{alertname="Watchdog",alertstate="firing",namespace="test3",severity="none",thanos_ruler_replica="thanos-ruler-user-workload-0"}	1
ALERTS{alertname="Watchdog",alertstate="firing",namespace="test3",severity="none",thanos_ruler_replica="thanos-ruler-user-workload-1"}	1


Version-Release number of selected component (if applicable):
4.5.0-0.nightly-2020-03-31-203533

How reproducible:
Always

Steps to Reproduce:
1. See the description
2.
3.

Actual results:
thanos_ruler_replica label is not dropped

Expected results:
should drop thanos_ruler_replica label

Additional info:

Comment 3 Junqi Zhao 2020-04-03 01:17:22 UTC
issue is fixed with 4.5.0-0.nightly-2020-04-02-195956

# oc -n openshift-monitoring get deploy thanos-querier -oyaml | grep "thanos_ruler_replica"
        - --query.replica-label=thanos_ruler_replica

search in thanos-querier
ALERTS{alertname="Watchdog",alertstate="firing",namespace="load"}

selected deduplication, result is
Element 	Value
ALERTS{alertname="Watchdog",alertstate="firing",namespace="load",severity="none"}	1

don't select deduplication, result is
Element 	Value
ALERTS{alertname="Watchdog",alertstate="firing",namespace="load",severity="none",thanos_ruler_replica="thanos-ruler-user-workload-0"}	1
ALERTS{alertname="Watchdog",alertstate="firing",namespace="load",severity="none",thanos_ruler_replica="thanos-ruler-user-workload-1"}	1

Comment 5 errata-xmlrpc 2020-07-13 17:24:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409


Note You need to log in before you can comment on or make changes to this bug.