Bug 2116644

Summary: Importer pod is failing to start with error "MountVolume.SetUp failed for volume "cdi-proxy-cert-vol" : configmap "custom-ca" not found"
Product: Container Native Virtualization (CNV) Reporter: nijin ashok <nashok>
Component: StorageAssignee: Arnon Gilboa <agilboa>
Status: CLOSED ERRATA QA Contact: Yan Du <yadu>
Severity: high Docs Contact:
Priority: high    
Version: 4.10.3CC: agilboa, alitke, awels, dbewley, jcall, mrashish, yadu
Target Milestone: ---   
Target Release: 4.12.0   
Hardware: All   
OS: Unspecified   
Whiteboard:
Fixed In Version: v4.12.0-72 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-01-24 13:39:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2155740    

Description nijin ashok 2022-08-09 06:24:45 UTC
Description of problem:

The cluster is having cluster proxy configured as below:

~~~
apiVersion: v1
items:
- apiVersion: config.openshift.io/v1
  kind: Proxy
....
....
  spec:
    trustedCA:
      name: custom-ca <<<
~~~

However, the custom-ca was only created in openshit-config namespace by following https://docs.openshift.com/container-platform/4.10/networking/enable-cluster-wide-proxy.html.

However, while importing the disk, the importer pod is in ContainerCreating status and we have the below events:

~~~
12m         Warning   FailedMount                   pod/importer-rhel7-sophisticated-parrotfish            MountVolume.SetUp failed for volume "cdi-proxy-cert-vol" : configmap "custom-ca" not found
~~~

This is because the custom-ca is not available in the namespace where we are importing the disk. The user has to manually copy the ConfigMap custom-ca to the namespace for the import to work. 

Version-Release number of selected component (if applicable):

OpenShift Virtualization   4.10.3

How reproducible:

100 %

Steps to Reproduce:

1. Add spec.trustedCA in the cluster-wide proxy configuration.
2. Create the ConfigMap that contains CA certificates in openshift-config namespace.
3. Try to import a image in namespace other than openshift-config. The importer pod will be stuck in `ContainerCreating` status.

Actual results:

Importer pod is failing to start with error "MountVolume.SetUp failed for volume "cdi-proxy-cert-vol" : configmap "custom-ca" not found"

Expected results:

Since the error is creating confusion, it would be ideal if the ConfigMap is automatically copied to the namespace where the user is trying to import the VM/disk. If not, I think we should mention this is the documentation to copy the ConfigMap manually.

Additional info:

Comment 1 Alexander Wels 2022-08-10 13:04:54 UTC
We had discussed this at some point, and come up with a plan to address the issue, just never got around to it. The idea is basically what you said, have the CDI controller automatically copy the config map into the namespace we are going to import in. Give it a random name, and make sure the config map is owned by the importer pod. That way once the importer pod is removed, the config map goes with it. This should not be too hard to implement.

Comment 3 Yan Du 2022-09-28 12:14:39 UTC
Alexander, do you plan to fix it in 4.10.6?

Comment 4 Alexander Wels 2022-09-28 12:16:24 UTC
We have a plan on how to fix it, just haven't scheduled time to do it. So no plans for 4.10.6

Comment 7 Arnon Gilboa 2022-12-21 13:43:10 UTC
Adam, Alexander, it's fixed on 4.12. Backport for 4.11 is pending for z-stream release.
Any reason to backport it to 4.10 as well (it's a bit harder as only part of the PR is relevant).

Comment 8 Alexander Wels 2022-12-27 01:04:00 UTC
I don't think so, I have no one requesting a backport to 4.10

Comment 10 Adam Litke 2023-01-10 13:24:39 UTC
Arnon, there seems to be a missing backport to 1.43.  Please create one and link to this BZ.  In the meantime I am moving this back to assigned.

Comment 11 Adam Litke 2023-01-10 13:43:31 UTC
We decided to fix this for 4.11 and 4.12 (not 4.10).  Retargeting.

Comment 12 Yan Du 2023-01-12 08:48:14 UTC
Verified on CNV v4.12.0-769, import succeed when proxy is set

$ oc get pod
NAME                READY   STATUS    RESTARTS   AGE
importer-fedora   1/1     Running   0          6s
$ oc get dv
NAME       PHASE              PROGRESS   RESTARTS   AGE
fedora     ImportInProgress   11.15%                13s
$ oc get pvc
NAME       STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS         AGE
fedora   Bound    pvc-9e6987bf-e7f9-4704-b3d7-0bdae9fb0a6c   149Gi      RWO            hostpath-csi-basic   83s

Comment 15 errata-xmlrpc 2023-01-24 13:39:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Virtualization 4.12.0 Images security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:0408