Bug 1788477

Summary:	proxy user-ca-bundle did not present in grafana pod /etc/pki/tls/certs/ca-bundle.crt
Product:	OpenShift Container Platform	Reporter:	Muhammad Aizuddin Zali <mzali>
Component:	Monitoring	Assignee:	Pawel Krupa <pkrupa>
Status:	CLOSED ERRATA	QA Contact:	Junqi Zhao <juzhao>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.2.0	CC:	alegrand, anpicker, erooth, kakkoyun, lcosic, mloibl, mzali, pkrupa, scuppett, surbania
Target Milestone:	---
Target Release:	4.2.z
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-01-22 10:46:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1776085
Bug Blocks:

Description Muhammad Aizuddin Zali 2020-01-07 10:19:21 UTC

Description of problem:
When replacing default ingress certificate with custom organization root CA, added root CA into proxies/config. Grafana pod did not take the root CA into it pod CA bundle. Hence grafana proxy POST call  auth URL returned below error:

2020/01/07 10:14:11 oauthproxy.go:645: error redeeming code (client:10.9.0.1:43806): Post https://oauth-openshift.apps.ocp4.local.bytewise.my/oauth/token: x509: certificate signed by unknown authority
2020/01/07 10:14:11 oauthproxy.go:438: ErrorPage 500 Internal Error Internal Error


Version-Release number of selected component (if applicable):
4.1.12

How reproducible:

Follow this guide[1] to add cluster wide new custom root CA and then replace default ingress certificate from here[2]. Login to the cluster and open up Grafana dashboard will shown above error.


[1]: https://docs.openshift.com/container-platform/4.2/networking/enable-cluster-wide-proxy.html
[2]: https://docs.openshift.com/container-platform/4.2/authentication/certificates/replacing-default-ingress-certificate.html

Steps to Reproduce:
1.
2.
3.

Actual results:
Grafana CA trust bundle did not include proxies certificate for trusting custom root CA.



Expected results:
Grafana CA trust bundle should include proxies certificate and trust the custom root CA.

Additional info:

Comment 1 Muhammad Aizuddin Zali 2020-01-07 10:23:44 UTC

1. The custom root CA:

mzali  ~  cat  CA-Auth/rootCA.crt  | head -n 2
-----BEGIN CERTIFICATE-----
MIIFxTCCA62gAwIBAgIUIQeGpl00JG5PC1Db0xrNpW9aiu0wDQYJKoZIhvcNAQEL

2. The trust bundle does not include the root CA:

mzali  ~  oc rsh grafana-6cc69d4d4b-5xc7b 
Defaulting container name to grafana.
Use 'oc describe pod/grafana-6cc69d4d4b-5xc7b -n openshift-monitoring' to see all of the containers in this pod.
sh-4.2$ cat /etc/pki/tls/certs/ca-bundle.crt | grep MIIFxTCCA62gAwIBAgIUIQeGpl00JG5PC1Db0xrNpW9aiu0wDQYJKoZIhvcNAQEL
sh-4.2$

Comment 2 Stephen Cuppett 2020-01-07 13:28:45 UTC

Setting target to active development branch (4.4). Fixes (if any) will result in cloned BZs for backports (where required/requested) for prior versions.

Comment 11 errata-xmlrpc 2020-01-22 10:46:40 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0107