Bug 1833438

Summary:	Logging metrics not showing up in Prometheus in 4.4 and 4.5
Product:	OpenShift Container Platform	Reporter:	Mike Fiedler <mifiedle>
Component:	Logging	Assignee:	Periklis Tsirakidis <periklis>
Status:	CLOSED WORKSFORME	QA Contact:	Anping Li <anli>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.4	CC:	aos-bugs, periklis
Target Milestone:	---
Target Release:	4.4.z
Hardware:	Unspecified
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-05-22 15:54:01 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1832125
Bug Blocks:

Description Mike Fiedler 2020-05-08 16:39:36 UTC

Description of problem:

ES and Fluentd metrics do not appear in Prometheus in 4.4 and 4.5.   Earlier in the 4.4 release they were showing up fine.   This is using the 4.4 GA operators for logging.

This script returns good data:  https://github.com/openshift/elasticsearch-operator/blob/master/hack/testing-olm/test-200-verify-es-metrics-access.sh

But now ES or fluentd metrics are in the Prometheus list and attempting to retrieve them by name fails.

Version-Release number of selected component (if applicable): 4.4.3 and 4.5.0-0.nightly-2020-05-08-095803 for the cluster
Cluster logging operator: 4.4.0-202005072005 
ES operator: 4.4.0-202004261927


How reproducible: Always, multiple installs are exhibiting this


Steps to Reproduce:
1. Install a 4.4 or 4.5 cluster
2. From Operator Hub, install the ES and clusterlogging operators and create a logging cluster
3. Go to prometheus and look for es* and fluentd* metrics in the list

Actual results:

metrics are not there

Expected results:

metrics available

Additional info:

Will add location of must-gather, let us know what other doc is needed.

Comment 3 Jeff Cantrill 2020-05-12 00:11:02 UTC

Setting to 4.4 found/target as its likely the same issue as 1832125

Comment 4 Periklis Tsirakidis 2020-05-22 09:28:50 UTC

@Mike

I cannot confirm this issue for 4.4.0-202005121717 and since the 4.5.0 parent issue is fixed can you check again?

Comment 5 Mike Fiedler 2020-05-22 15:54:01 UTC

Working OK for me in 4.4.0.   Closing as notabug.   4.5 verification blocked by https://bugzilla.redhat.com/show_bug.cgi?id=1838929 right now