Bug 1855220

Summary:	Kibana OAuth HTTP 500 error - x509: certificate signed by unknown authority
Product:	OpenShift Container Platform	Reporter:	Fatima <fshaikh>
Component:	Logging	Assignee:	Periklis Tsirakidis <periklis>
Status:	CLOSED ERRATA	QA Contact:	Qiaoling Tang <qitang>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	4.4	CC:	anli, aos-bugs, jcantril, jmalde, mfuruta, periklis
Target Milestone:	---
Target Release:	4.4.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-08-04 14:16:01 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1834311
Bug Blocks:

Description Fatima 2020-07-09 09:34:33 UTC

Description of problem:

The bug: Kibana OAuth HTTP 500 error - x509: certificate signed by unknown authority was solved in OCP 4.4.8 but one of the customer is facing this issue is OCP 4.4.10

Workaround provided in https://access.redhat.com/solutions/5000761 works here.


Will add the pod logs and cm here in private comments.

Comment 3 Periklis Tsirakidis 2020-07-10 11:01:05 UTC

@fatima

Please provide a logging-dump with [1]

[1] https://github.com/openshift/origin-aggregated-logging/blob/master/hack/logging-dump.sh

Comment 5 Periklis Tsirakidis 2020-07-20 12:17:30 UTC

@Fatima

Thank you for raising this issue in a separate BZ. Unfortunately the backport to 4.4 was half-done due to an internal change of kibana reconciliation between 4.5 and 4.4. In detail, until 4.4 kibana was reconciled by cluster-logging-operator. From 4.5, this responsibility was moved w/o breaking user experience or active migrations to elasticsearch-operator. Thus, it was all to easy to miss to backport the fix on the right operator for 4.4.

To check this, you should see on the customer cluster a reconciled CA bundle for fluentd (half of successful backport) but not for kibana (missing part in backport), e.g.:

Fluentd:
 - mountPath: /etc/pki/ca-trust/extracted/pem/
   name: fluentd-trusted-ca-bundle
   readOnly: true

Kibana is missing a similar entry according to the provided logging dump.

This is working for 4.5 onwards. The provided PR takes care to fix this also for 4.4.z.

Comment 8 Qiaoling Tang 2020-07-27 06:36:38 UTC

Verified with clusterlogging.4.4.0-202007240028.p0

        volumeMounts:
        - mountPath: /secret
          name: kibana-proxy
          readOnly: true
        - mountPath: /etc/pki/ca-trust/extracted/pem/
          name: kibana-trusted-ca-bundle
          readOnly: true

Comment 10 errata-xmlrpc 2020-08-04 14:16:01 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.4.15 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3128