Bug 2225116

+++ This bug was initially created as a clone of Bug #2156525 +++

Description of the problem:
VMExport: can't download a PVC that was created from DV on NFS (when there's no VM that owns this DV/PVC)

Version-Release number of selected component (if applicable):
4.12, 4.13, 4.14

How reproducible:
Always on NFS, when DV/PVC is not owned by a VM
It works with NFS when VM owns the DV/PVC
It works with HPP and OCS with no regard to a VM

Steps to Reproduce:
1. Create an NFS DV
2. Create a VMExport for PVC
3. See the VMExport is Ready
    $ oc get vmexport
    NAME             SOURCEKIND              SOURCENAME   PHASE
    export-pvc-nfs   PersistentVolumeClaim   dv-source    Ready

4. Try to download the image, but get an error
    $ virtctl vmexport download export-pvc-nfs --output=disk-pvc-nfs.img --keep-vme
    Processing completed successfully
    Bad status: 500 Internal Server Error

5. See the log:

    $ oc logs virt-export-export-pvc-nfs | grep error
{"component":"virt-exportserver-virt-export-export-pvc-nfs","level":"error","msg":"error opening /export-volumes/dv-source/disk.img","pos":"exportserver.go:315","reason":"open /export-volumes/dv-source/disk.img: permission denied","timestamp":"2022-12-27T09:22:32.371279Z"}


Actual results:
Bad status: 500 Internal Server Error

Expected results:
Image downloaded successfully


Additional info:

$ cat dv-nfs.yaml

apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
  name: dv-source
spec:
  source:
      http:
         url: <cirros.img>
  storage:
    resources:
      requests:
        storage: 1Gi
    storageClassName: nfs
    volumeMode: Filesystem


$ cat vmexport-dv-nfs.yaml 

apiVersion: export.kubevirt.io/v1alpha1
kind: VirtualMachineExport
metadata:
  name: export-pvc-nfs
spec:
  source:
    apiGroup: ""
    kind: PersistentVolumeClaim
    name: dv-source

--- Additional comment from Alex Kalenyuk on 2022-12-27 10:33:23 UTC ---

This happens because NFS does not support fsGroup
https://kubernetes.io/blog/2020/12/14/kubernetes-release-1.20-fsgroupchangepolicy-fsgrouppolicy/#allow-csi-drivers-to-declare-support-for-fsgroup-based-permissions

[virt-exportserver@virt-export-export-pvc-nfs ~]$ id
uid=1001(virt-exportserver) gid=1001(virt-exportserver) groups=1001(virt-exportserver),107
[virt-exportserver@virt-export-export-pvc-nfs ~]$ ls -la /export-volumes/dv-source/disk.img
-rw-rw----. 1 107 99 1073741824 Dec 27 09:17 /export-volumes/dv-source/disk.img
[virt-exportserver@virt-export-export-pvc-nfs ~]$ md5sum /export-volumes/dv-source/disk.img
md5sum: /export-volumes/dv-source/disk.img: Permission denied

I am not sure if there's something we could do here, seeing as we want to run non root (restricted PSA),
and that requires fsGroup.

While debugging this, I did notice that we don't set runAsGroup in the importer manifests/Dockerfile, which is something we might want to reconsider
as it could somehow mitigate this?
$ oc exec importer-windows-ey0bvq-installcdrom -it bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
bash-5.1$ id
uid=107(107) gid=0(root) groups=0(root),107
bash-5.1$ ls -la /data/disk.img 
-rw-r--r--. 1 107 99 5694537728 Dec 27 10:30 /data/disk.img
@mhenriks WDYT?

--- Additional comment from Michael Henriksen on 2022-12-29 01:49:52 UTC ---

Export pod should run as run as user 107.

--- Additional comment from Alexander Wels on 2023-02-06 15:15:07 UTC ---

So I checked https://github.com/kubevirt/kubevirt/blob/main/cmd/virt-launcher/BUILD.bazel#L111-L118 against https://github.com/kubevirt/kubevirt/blob/main/cmd/virt-exportserver/BUILD.bazel#L32-L39 and it doesn't look like the virt-export server is running as 107, but 1001.