Bug 1995937

Summary:	Cluster-wide proxy settings not picked up by insights operator.
Product:	OpenShift Container Platform	Reporter:	harsharm
Component:	Insights Operator	Assignee:	Tomas Remes <tremes>
Status:	CLOSED NOTABUG	QA Contact:	Dmitry Misharov <dmisharo>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	4.6	CC:	aos-bugs, inecas, mklika, tremes
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	2003657 (view as bug list)		Environment:
Last Closed:	2021-10-04 09:56:22 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2003657

Description harsharm 2021-08-20 08:55:08 UTC

Description of problem:

Insights operator is in the state DEGRADED when there is a cluster-wide proxy configured. The below error messages can be seen in the pod logs.

~~~
The operator has some internal errors: Unable to report: unable to build request to connect to Insights server: Post "https://cloud.redhat.com/api/ingress/v1/upload": x509: certificate signed by unknown authority
~~~

Version-Release number of selected component (if applicable):
OCP 4.6

Actual results:
The insights operator becomes degraded.

Expected results:
Insights Operator to take the cluster-wide proxy CA into consideration.

Additional info:

When we make the Operator aware about the CA, by updating the CA explicitly, and then try curl --cacert /var/run/configmaps/trusted-ca-bundle/ca-bundle.crt https://cloud.redhat.com from the Insights Operator pod it works. The insights operator also comes out of degraded state.

Comment 1 Tomas Remes 2021-08-20 10:45:23 UTC

The Insights Operator uploads an archive to the ingress service url (mentioned above) and if the upload fails more than 5 times in a row then the operator is marked as degraded. Insights Operator doesn't ignore cluster-wide proxy settings. The settings is "injected" into the Insights Operator container via environment variables (HTTP_PROXY, HTTPS_PROXY, NO_PROXY). Probably the insights-operator is not adding the trustedCA (defined in the cluster proxy) to the root CAs when doing the request. This needs to be investigated and possibly fixed!

Comment 2 Tomas Remes 2021-08-23 14:02:21 UTC

Sorry for the possibly recurring question, but can you @harsharm please confirm that this is basically a request to read and respect the CA (defined via "trustedCA" in cluster-wide proxy) in the Insights Operator? Maybe it would be also good to mention some workaround (if any).

Comment 3 harsharm 2021-08-24 09:56:09 UTC

Yes that is correct, when we make the Operator aware about the CA, by updating the CA explicitly, and then try curl --cacert /var/run/configmaps/trusted-ca-bundle/ca-bundle.crt https://cloud.redhat.com from the Insights Operator pod it works. The insights operator also comes out of degraded state.

Comment 4 Tomas Remes 2021-10-04 09:56:22 UTC

I can't reproduce this problem. Note that the Insights operator always looks at the data in the corresponding config map containing also the user defined CA cert and thus the certificate should be respected. See https://github.com/openshift/insights-operator/blob/master/pkg/insights/insightsclient/insightsclient.go#L106