Created attachment 1803996 [details] Watchdog details page Description of problem: go to "Observe -> Alerting", click "Watchdog" to check its details, see from the picture, the labels are: Labels alertname=Watchdog severity=none searched "ALERTS{alertname="Watchdog"}" from API, it has another label which key/value is "prometheus": "openshift-monitoring/k8s" # token=`oc sa get-token prometheus-k8s -n openshift-monitoring` # oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://thanos-querier.openshift-monitoring.svc:9091/api/v1/query?query=ALERTS%7Balertname%3D%22Watchdog%22%7D' | jq { "status": "success", "data": { "resultType": "vector", "result": [ { "metric": { "__name__": "ALERTS", "alertname": "Watchdog", "alertstate": "firing", "prometheus": "openshift-monitoring/k8s", "severity": "none" }, "value": [ 1626858123.199, "1" ] } ] } } Version-Release number of selected component (if applicable): 4.9.0-0.nightly-2021-07-20-221331 How reproducible: always Steps to Reproduce: 1. see the description 2. 3. Actual results: Expected results: Additional info:
That page takes its input from the /rules endpoint rather than the /query endpoint, so I believe the behavior is expected. I'm therefore closing as NOTABUG, but please reopen if more discussion is required.