Bug 1731006 - Prometheus metrics for ES are unavailable after plugin update to 5.6.13.6
Summary: Prometheus metrics for ES are unavailable after plugin update to 5.6.13.6
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.1.z
Assignee: Jeff Cantrill
QA Contact: Anping Li
URL:
Whiteboard:
Depends On: 1729297
Blocks: 1728856
TreeView+ depends on / blocked
 
Reported: 2019-07-18 06:36 UTC by Jeff Cantrill
Modified: 2020-01-27 13:56 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: The authentication class is instantiated multiple times with and without the desired configuration Consequence: User's are denied access to metrics Fix: Add SAR config to the authentication_backend Result: Multi-tenant plugin executes SARs and allows access to metrics
Clone Of: 1729297
Environment:
Last Closed: 2019-07-31 02:44:57 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift origin-aggregated-logging pull 1697 0 'None' closed [release-4.1] Bug 1731006: Add authentication config to properly check SAR for addi… 2020-01-27 13:55:34 UTC
Red Hat Product Errata RHBA-2019:1866 0 None None None 2019-07-31 02:45:00 UTC

Description Jeff Cantrill 2019-07-18 06:36:48 UTC
+++ This bug was initially created as a clone of Bug #1729297 +++

+++ This bug was initially created as a clone of Bug #1728856 +++

Description of problem:
Updating to the latest version of the multi-tenant plugin blocks viewing by metrics unless you have cluster admin rights

Version-Release number of selected component (if applicable):


How reproducible:
Always


Steps to Reproduce:
1. Deploy logging
2. Set up an SA that can scrape metrics
3. curl -kv https://172.30.59.79/_prometheus/metrics -H"Authorization: Bearer $(oc serviceaccounts get-token metrics-test)"


Actual results: 403


Expected results: 200


Additional info: https://github.com/openshift/origin-aggregated-logging/issues/1686

Comment 2 Anping Li 2019-07-25 05:01:42 UTC
The sa prometheus-k8s can fetch metrics from es. 
sh-4.2# curl -k -H "Authorization: Bearer $token" -H "Content-type: application/json"  https://172.30.119.95:60000/_prometheus/metrics
# HELP es_index_segments_memory_bytes Memory used by segments
# TYPE es_index_segments_memory_bytes gauge
es_index_segments_memory_bytes{cluster="elasticsearch",type="points",index=".kibana",context="total",} 0.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="norms",index=".operations.2019.07.25",context="primaries",} 47552.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="indexwriter",index=".kibana",context="primaries",} 0.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="docvalues",index=".searchguard",context="primaries",} 636.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="termvectors",index=".kibana",context="total",} 0.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="versionmap",index=".operations.2019.07.25",context="total",} 63240.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="terms",index=".operations.2019.07.25",context="primaries",} 537727.0
es_index_segments_memory_bytes{cluster="elasticsearch",type="indexwriter",index=".kibana.647a750f1787408bf50088234ec0edd5a6a9b2ac

Comment 4 Anping Li 2019-07-30 03:01:00 UTC
Also verified using registry.redhat.io/openshift4/ose-logging-elasticsearch5:v4.1.4-201906271212.

Comment 6 errata-xmlrpc 2019-07-31 02:44:57 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1866


Note You need to log in before you can comment on or make changes to this bug.