Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1889006

Summary: cvo downloading update graph is not using (ignoring) installed pki root certificates
Product: OpenShift Container Platform Reporter: Kai-Uwe Rommel <kai-uwe.rommel>
Component: Cluster Version OperatorAssignee: Over the Air Updates <aos-team-ota>
Status: CLOSED DUPLICATE QA Contact: Johnny Liu <jialiu>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.5CC: aos-bugs, jokerman, wking
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-16 20:00:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
screen snapshot of error none

Description Kai-Uwe Rommel 2020-10-16 18:40:35 UTC 
Created attachment 1722136 [details]
screen snapshot of error

Description of problem:
I installed a cluster in an environment where there is a transparent inspecting/filtering web proxy integrated in the default gateway (firewall). This proxy does TLS termination and re-encryption so when the cluster installation downloads packages, the files retrieved are signed by a certificate created by an internal corporate CA. To handle this, I installed that corporate CA's root certificate into the local CoreOS instances with the initial ignition. This worked fine and the installation went through successfully. For previous installations in that environment we had to ask the network admins to set an exception and let anything through from quay.io etc. unfiltered, what they did not like, obviously. So this is a solution to this problem.
However, now that the cluster is up and running, it can't check for updates. The CVO complains "Unable to retrieve updates: Get ... signed by unknown authority". Apparently, the cluster operator that is downloading the update graph is ignoring the root certificates in /etc/pki/ca-trust in the node's OS. 

Version-Release number of the following components:
OpenShift 4.5.x

How reproducible:
always

Steps to Reproduce:
See above.

Actual results:

Expected results:

Additional info:
See Github issue: https://github.com/openshift/okd/issues/326
Same issue in OKD and OCP.
Comment 1 W. Trevor King 2020-10-16 20:00:52 UTC 
> Apparently, the cluster operator that is downloading the update graph is ignoring the root certificates in /etc/pki/ca-trust in the node's OS. 

Yeah, CVO's trust store is isolated from the node's.  To configure the CVO's trust store, set trustedCA in the Proxy object [1].  Up through 4.5, the CVO needed both trustedCA and httpsProxy set there in order to consume the trustedCA store.  With 4.6 and later, the CVO will consume the trustedCA store regardless of the presence of httpProxy or httpsProxy [2].  Also some related discussion in bug 1773419, which seems like this use case, so I'm closing this one as a dup.  Feel free to add further comments to this bug if it doesn't seem like a dup to you.

[1]: https://docs.openshift.com/container-platform/4.5/networking/enable-cluster-wide-proxy.html#nw-proxy-configure-object_config-cluster-wide-proxy
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=1797123#c15

*** This bug has been marked as a duplicate of bug 1773419 ***
Comment 2 Kai-Uwe Rommel 2020-10-18 19:02:47 UTC 
Yes, it indeed is a duplicate of bug 1773419.
Please see my note there.
I would expect that it could be easily fixed ... the reporter there already reports that there is a solution which sounds pretty much like a one-liner.