Bug 1731006

Summary:	Prometheus metrics for ES are unavailable after plugin update to 5.6.13.6
Product:	OpenShift Container Platform	Reporter:	Jeff Cantrill <jcantril>
Component:	Logging	Assignee:	Jeff Cantrill <jcantril>
Status:	CLOSED ERRATA	QA Contact:	Anping Li <anli>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	4.1.0	CC:	anli, aos-bugs, rmeggins
Target Milestone:	---
Target Release:	4.1.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: The authentication class is instantiated multiple times with and without the desired configuration Consequence: User's are denied access to metrics Fix: Add SAR config to the authentication_backend Result: Multi-tenant plugin executes SARs and allows access to metrics	Story Points:	---
Clone Of:	1729297	Environment:
Last Closed:	2019-07-31 02:44:57 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1729297
Bug Blocks:	1728856

Description Jeff Cantrill 2019-07-18 06:36:48 UTC

+++ This bug was initially created as a clone of Bug #1729297 +++

+++ This bug was initially created as a clone of Bug #1728856 +++

Description of problem:
Updating to the latest version of the multi-tenant plugin blocks viewing by metrics unless you have cluster admin rights

Version-Release number of selected component (if applicable):


How reproducible:
Always


Steps to Reproduce:
1. Deploy logging
2. Set up an SA that can scrape metrics
3. curl -kv https://172.30.59.79/_prometheus/metrics -H"Authorization: Bearer $(oc serviceaccounts get-token metrics-test)"


Actual results: 403


Expected results: 200


Additional info: https://github.com/openshift/origin-aggregated-logging/issues/1686

Comment 2 Anping Li 2019-07-25 05:01:42 UTC

The sa prometheus-k8s can fetch metrics from es. 
sh-4.2# curl -k -H "Authorization: Bearer $token" -H "Content-type: application/json"  https://172.30.119.95:60000/_prometheus/metrics
# HELP es_index_segments_memory_bytes Memory used by segments
# TYPE es_index_segments_memory_bytes gauge
es_index_segments_memory_bytes{cluster="elasticsearch",type="points",index=".kibana",context="total",} 0.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="norms",index=".operations.2019.07.25",context="primaries",} 47552.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="indexwriter",index=".kibana",context="primaries",} 0.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="docvalues",index=".searchguard",context="primaries",} 636.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="termvectors",index=".kibana",context="total",} 0.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="versionmap",index=".operations.2019.07.25",context="total",} 63240.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="terms",index=".operations.2019.07.25",context="primaries",} 537727.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="indexwriter",index=".kibana.647a750f1787408bf50088234ec0edd5a6a9b2ac

Comment 4 Anping Li 2019-07-30 03:01:00 UTC

Also verified using registry.redhat.io/openshift4/ose-logging-elasticsearch5:v4.1.4-201906271212.

Comment 6 errata-xmlrpc 2019-07-31 02:44:57 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1866