Description of problem: Insights operator is in the state DEGRADED when there is a cluster-wide proxy configured. The below error messages can be seen in the pod logs. ~~~ The operator has some internal errors: Unable to report: unable to build request to connect to Insights server: Post "https://cloud.redhat.com/api/ingress/v1/upload": x509: certificate signed by unknown authority ~~~ Version-Release number of selected component (if applicable): OCP 4.6 Actual results: The insights operator becomes degraded. Expected results: Insights Operator to take the cluster-wide proxy CA into consideration. Additional info: When we make the Operator aware about the CA, by updating the CA explicitly, and then try curl --cacert /var/run/configmaps/trusted-ca-bundle/ca-bundle.crt https://cloud.redhat.com from the Insights Operator pod it works. The insights operator also comes out of degraded state.
The Insights Operator uploads an archive to the ingress service url (mentioned above) and if the upload fails more than 5 times in a row then the operator is marked as degraded. Insights Operator doesn't ignore cluster-wide proxy settings. The settings is "injected" into the Insights Operator container via environment variables (HTTP_PROXY, HTTPS_PROXY, NO_PROXY). Probably the insights-operator is not adding the trustedCA (defined in the cluster proxy) to the root CAs when doing the request. This needs to be investigated and possibly fixed!
Sorry for the possibly recurring question, but can you @harsharm please confirm that this is basically a request to read and respect the CA (defined via "trustedCA" in cluster-wide proxy) in the Insights Operator? Maybe it would be also good to mention some workaround (if any).
Yes that is correct, when we make the Operator aware about the CA, by updating the CA explicitly, and then try curl --cacert /var/run/configmaps/trusted-ca-bundle/ca-bundle.crt https://cloud.redhat.com from the Insights Operator pod it works. The insights operator also comes out of degraded state.
I can't reproduce this problem. Note that the Insights operator always looks at the data in the corresponding config map containing also the user defined CA cert and thus the certificate should be respected. See https://github.com/openshift/insights-operator/blob/master/pkg/insights/insightsclient/insightsclient.go#L106