Bug 1828198

Summary:	CDI: scratch space looks for default storage class even when scratchSpaceStorageClass is configured
Product:	Container Native Virtualization (CNV)	Reporter:	Natalie Gavrielov <ngavrilo>
Component:	Storage	Assignee:	Alexander Wels <awels>
Status:	CLOSED NOTABUG	QA Contact:	Natalie Gavrielov <ngavrilo>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	2.3.0	CC:	alitke, cnv-qe-bugs, ncredi
Target Milestone:	---	Keywords:	Automation
Target Release:	2.4.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:	Cause: scratchSpaceStorageClass is defined to be hostpath-provisioner while hostpath-provisioner is not set as default storage Consequence: upload pod doesnt become ready Workaround (if any): Set hostpath-provisioner Storage class to be default Result:	Story Points:	---
Clone Of:
Clones:	1828265 (view as bug list)		Environment:
Last Closed:	2020-05-20 11:23:47 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1828265

Description Natalie Gavrielov 2020-04-27 09:56:00 UTC

Description of problem:
Seems that even though scratchSpaceStorageClass is defined, scratch space required operations look for a default storage class to use.

Version-Release number of selected component:
CNV 2.3

How reproducible:
Not sure yet.

Steps to Reproduce:
1. Have one hostpath-provisioner storage class in the environment
2. The storage class is not defined as default (no default storage class)
3. scratchSpaceStorageClass is defined to be hostpath-provisioner
4. upload a data volume:
virtctl image-upload dv dv-name --size=1Gi --image-path=cirros-0.4.0-x86_64-disk.qcow2 --insecure

Actual results:
DataVolume default/dv-name created
Waiting for PVC dv-name upload pod to be ready...
timed out waiting for the condition

Expected results:
Upload to succeed 

Additional info:

After upload failed, I set a default storage class:
$ oc patch storageclass hostpath-provisioner -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "true"}}}'
storageclass.storage.k8s.io/hostpath-provisioner patched

oc get sc
NAME                             PROVISIONER                        RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
hostpath-provisioner (default)   kubevirt.io/hostpath-provisioner   Delete          WaitForFirstConsumer   false                  2d18h

and tried to upload again:
$ virtctl image-upload dv dv-name --size=1Gi --image-path=cirros-0.4.0-x86_64-disk.qcow2 --insecure
DataVolume default/dv-name created
Waiting for PVC dv-name upload pod to be ready...
Pod now ready
Uploading data to https://cdi-uploadproxy-openshift-cnv.apps.bm-rhel.oc4

 12.13 MiB / 12.13 MiB [===============================================================================================================================================] 100.00% 0s

Uploading data completed successfully, waiting for processing to complete, you can hit ctrl-c without interrupting the progress
Processing completed successfully
Uploading cirros-0.4.0-x86_64-disk.qcow2 completed successfully

$ oc get pvc
NAME      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS           AGE
dv-name   Bound    pvc-ac2cc14e-2999-4546-8f78-1f7788c1c2f3   916Gi      RWO            hostpath-provisioner   13s

$ oc get dv
NAME      PHASE       PROGRESS   AGE
dv-name   Succeeded              1m


Output for the failed test:
-------------------------------------------------------------------------

$ oc get sc
NAME                   PROVISIONER                        RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
hostpath-provisioner   kubevirt.io/hostpath-provisioner   Delete          WaitForFirstConsumer   false                  2d17h

-------------------------------------------------------------------------

$ oc get pods
NAME                 READY   STATUS    RESTARTS   AGE
cdi-upload-dv-name   0/1     Pending   0          14s

-------------------------------------------------------------------------

$ oc get pod cdi-upload-dv-name -oyaml
apiVersion: v1
kind: Pod
metadata:
  annotations:
    cdi.kubevirt.io/storage.createdByUploadController: "yes"
  creationTimestamp: "2020-04-27T07:46:04Z"
  labels:
    app: containerized-data-importer
    cdi.kubevirt.io: cdi-upload-server
    service: cdi-upload-dv-name
  name: cdi-upload-dv-name
  namespace: default
  ownerReferences:
  - apiVersion: v1
    blockOwnerDeletion: true
    controller: true
    kind: PersistentVolumeClaim
    name: dv-name
    uid: f8e09ecc-7f20-439f-bafe-d5caf7e85a1e
  resourceVersion: "2723602"
  selfLink: /api/v1/namespaces/default/pods/cdi-upload-dv-name
  uid: d3479bd3-4f0e-4132-8a52-440138ef67d8
spec:
  containers:
  - args:
    - -v=1
    env:
    - name: TLS_KEY
      value: |
        -----BEGIN RSA PRIVATE KEY-----
        MIIEpAIBAAKCAQEA4g2yDfDqdf8K44NLDDMjqqMGEfxUxuCgpi8tCYQn8XpqDo3P
        Z751ELYggPIEucjSO5FlKt828LcJPr2jD8cDHPySHf5qMYhzrzsajN4t/lYV6/tb
        1Eed5VajW3OZubc3hlSqMSD3VtpopOruSdHn4Vg1WUz3gf31u05TZ55YD8+FP0JG
        I8yNbggFRJSSNDJ5ssbk4FM5tCkUOs7M9LeR01GTi4KMOx7TVYqwr4FS8mGTDZN1
        dbJzW7jZuNjLs1+MzsNRZAj33hxr7XAHCpGctdSiqwopS74bDEkqDwi9h1cSmskY
        2xt9zUAP+RHrrUvDv4dy7V9rOG5AE+pCzkVMxwIDAQABAoIBAHRTceYkrLLN7Q+J
        F8yOYHNF8zC8NfAl1eXO7xEvldJ5GSymIy4S33EbF/zjDjWAt1b6CiKWvVIADtzf
        f5L16Y84FgUjUAhvXIC3HIGTL/3q1Nsb/XRu6j0DaYr9IQVl2EfjN7WSblWejS0Y
        zEjmoLngx4tNxSx6wMKDmv3+iPiQ7M06HM1KSwstA5xQjtmcK+2crJPVq44rk8fS
        wVGvLv0qFn8v+jwMW/7Fd+mNhAdvP9IofFvfk4d/wFV1Wm8T79F+0U9s57lPqeJr
        /QPa0q+0unO0vyaN2ig3knLVKQ+NyiL2aY+Hg4yrVKJaRA5f7ldTfiQyRl7ClVAV
        NVxEyxkCgYEA4qhop+lAuPdx8sFDhL1kV73S40aSBnOG7v3C+8Mev3HirsKLE/12
        MU6/sLiA74IOwR+NmrwMRHdQI8pZrllJMhDrK+kZaje9bOcfIuXs1XI/7kWV2pLR
        4O53MdstijUQoLEZWfMfHD/q6T68H24168ZYcXRzSLJltAPoz1hV2F0CgYEA/1FC
        GQWq/Ne/hQ5FzUCNtJMvwni5puhAqih2oklrsmm+mYkGlGziXFWW8crrH0YQOVQj
        whzx69OtP9UN9ZSUS1oLC1P6ZFD4t1Ygb4ytSokQYWkJIkx1KtnQ8oOT8FO71tOx
        F8mTluBJL5TGzoLfEBzwBXHYOElBD2TErPcL13MCgYEAwQ/v7S6KmcLzxXIHX/au
        lOo0yrCey232PnXacvZ1p16LVo6RRpq75S4Nq6rM8LXYeXm19Vo+HMeAdKv0jcSx
        yzqMBqALrOUDrdKEPAabIAisGfodtcsfiLe2wbnO5Y1jt2yCipLDKh3rq3draF5J
        Aaa2SRWG6SJD7PzzfkHhBWUCgYBgBubuVFRoPpkY6TJVrWbeuNEoupFQ9l+D4C4m
        gajQ9LasNn5UCtPkWIiqLmDaZVaqyTudwZc31Gh/7pkOebCA2r/2pcIyQsYY5nrS
        mb1/oCeaC5BU5x7WXBAbdmOI62Vluj8MLFOnVnoQPEu91jkhbVDVK2lgWiTvLj5O
        s6gvgwKBgQCFBXqvWD23Nrgd5W7a9uYDX8zXIJmphpnqZNmg/OefseNSSglD0hMA
        QS56B49bWEDEC+GdQ7ZLWYj4S4bgqihuqgNEB1VH15Z0GK3tBS08eXEhSWYzd3I1
        VUNDrRYeGZK4BCjK0r0a1K0zasceliH5O+pFAXfgQHepANG+hKd+zA==
        -----END RSA PRIVATE KEY-----
    - name: TLS_CERT
      value: |
        -----BEGIN CERTIFICATE-----
        MIIDrDCCApSgAwIBAgIIN3h3XCb+vRUwDQYJKoZIhvcNAQELBQAwOzE5MDcGA1UE
        Awwwb3BlbnNoaWZ0LWNudl9jZGktdXBsb2Fkc2VydmVyLXNpZ25lckAxNTg3NzM3
        Mjg3MB4XDTIwMDQyNzA3NDYwM1oXDTIxMDQyNzA3NDYwNFowHTEbMBkGA1UEAxMS
        Y2RpLXVwbG9hZC1kdi1uYW1lMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC
        AQEA4g2yDfDqdf8K44NLDDMjqqMGEfxUxuCgpi8tCYQn8XpqDo3PZ751ELYggPIE
        ucjSO5FlKt828LcJPr2jD8cDHPySHf5qMYhzrzsajN4t/lYV6/tb1Eed5VajW3OZ
        ubc3hlSqMSD3VtpopOruSdHn4Vg1WUz3gf31u05TZ55YD8+FP0JGI8yNbggFRJSS
        NDJ5ssbk4FM5tCkUOs7M9LeR01GTi4KMOx7TVYqwr4FS8mGTDZN1dbJzW7jZuNjL
        s1+MzsNRZAj33hxr7XAHCpGctdSiqwopS74bDEkqDwi9h1cSmskY2xt9zUAP+RHr
        rUvDv4dy7V9rOG5AE+pCzkVMxwIDAQABo4HRMIHOMA4GA1UdDwEB/wQEAwIFoDAT
        BgNVHSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB0GA1UdDgQWBBSVExQh
        m5O83tkSXOUeOg58kEGMJjAfBgNVHSMEGDAWgBRHbM5Bq62iqBRorkyPYUDMQx1R
        6zBZBgNVHREEUjBQghJjZGktdXBsb2FkLWR2LW5hbWWCGmNkaS11cGxvYWQtZHYt
        bmFtZS5kZWZhdWx0gh5jZGktdXBsb2FkLWR2LW5hbWUuZGVmYXVsdC5zdmMwDQYJ
        KoZIhvcNAQELBQADggEBAOo1AUvlGIoOUS9YDralSYrZfFMJi579Vm3n7XgyMibH
        NPa82ykpaTyztAbb9a4rkcz3yoebmFHVIsi9EAxO/j8RlGs8RSDn11SHQN/Ozjt3
        kS5e+YkePHbEUjiaYqhb04KY8rjsOy6dQvVJ1dk4p4SYiQEr6Lf+1tXqgFxHZezv
        NNc7W24Pb6qJBbVoRbT8tQQY5GX7MRiRAhzF3/9mv0vrI2WVu3vEMud8i2yCfNxe
        XwRNwz7hUGgbLV0GBQlCsF994RAV+3EvfPipW9LQoP6bb7A9Xc09ejcCmi18MBHZ
        hDLNd7zD+OIBciNZNcXmsimQIuP9k8ho9JuCpKT9Pes=
        -----END CERTIFICATE-----
        -----BEGIN CERTIFICATE-----
        MIIDVDCCAjygAwIBAgIBATANBgkqhkiG9w0BAQsFADA7MTkwNwYDVQQDDDBvcGVu
        c2hpZnQtY252X2NkaS11cGxvYWRzZXJ2ZXItc2lnbmVyQDE1ODc3MzcyODcwHhcN
        MjAwNDI0MTQwODA2WhcNMzAwNDIyMTQwODA3WjA7MTkwNwYDVQQDDDBvcGVuc2hp
        ZnQtY252X2NkaS11cGxvYWRzZXJ2ZXItc2lnbmVyQDE1ODc3MzcyODcwggEiMA0G
        CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDtIynMFcz2Nv0QXRIxnMEB+MA1owji
        iuyhr0cjp2HzcmkpFMLjxnS4v75h9UZ78w4vHO+gECB/p5k6GIcxV81HeMlDRI00
        F4ELs5+hEmCbqJgb0PsHCCwvY+t7allDHJApCfnPS8FGVONJ9Zi22FG3bEMMbU+U
        8TCHUVSQrKyg05HlfXUjDEt3I73mJx4ytmnvl1HxyHpW2iClZKD36VLUHf96DZrh
        jgUHdgCVJa5Qc/anS1+dsyYzdRR+O4uD3uz79NbCRgw9RXigbQhOAwksz0BRR0OG
        JJvvbI9SPW634hZAoes22i8f2DdCtdlqfOuOyhM+xICOn3ccPdgeTaVbAgMBAAGj
        YzBhMA4GA1UdDwEB/wQEAwICpDAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBRH
        bM5Bq62iqBRorkyPYUDMQx1R6zAfBgNVHSMEGDAWgBRHbM5Bq62iqBRorkyPYUDM
        Qx1R6zANBgkqhkiG9w0BAQsFAAOCAQEAfuryljz1+9t67SsrN94Np7CIDdcVfSAi
        cekTM68t8JixqduRZdpj2T4zK8396pIUbAJ2GRal6U/z/EyF+dnLzAKPnGey+7mc
        rMqeQxJyrud4z2P+HYDAlcyXjQtGjcWPtxKW1+4vPNBw8OzRfXlE9GOhaZ1V80EX
        q7cqdSgSzZtpXAFxZK5T2NHOUAZ+U4kROpsPHTx3+kzKMWfLEUr8SYsBEvHwqOkq
        JjwXSNmJ+R9JeQhO3Q2apEUDR/NvmuD5NFRsnTG1HOGND6htfsiYKLpfNcM4GtnU
        na9ybWQgws692X0Cz1NIpW/sZahQD6yUNseXFhJYJufBjagkPMRYxw==
        -----END CERTIFICATE-----
    - name: CLIENT_CERT
      value: |
        -----BEGIN CERTIFICATE-----
        MIIDYjCCAkqgAwIBAgIBATANBgkqhkiG9w0BAQsFADBCMUAwPgYDVQQDDDdvcGVu
        c2hpZnQtY252X2NkaS11cGxvYWRzZXJ2ZXItY2xpZW50LXNpZ25lckAxNTg3NzM3
        Mjg3MB4XDTIwMDQyNDE0MDgwNloXDTMwMDQyMjE0MDgwN1owQjFAMD4GA1UEAww3
        b3BlbnNoaWZ0LWNudl9jZGktdXBsb2Fkc2VydmVyLWNsaWVudC1zaWduZXJAMTU4
        NzczNzI4NzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANhK/iX1dU7o
        BTeJBbRywHhKllA4X5t7m4Xqxp9gSXd6znNG8wtL0OnCNalEFQ30QWxABGIItr12
        rKFZTMhMI9gy/u03KHS9ovLfj/vkT1MGBQC8yQ+noG0JJqjuLyPO1zLAYRYYvmb5
        Aau8W9tSO18yJ5s3j4R/j+qt7MPj9CD1upPz/LDOQ0Zcs3foLIqG5mqG6Sm24+m7
        h6ZXEim13f3WgI2y8BOEAzUsgawCOrJLXeG5Ck3HetfmI55rOOXtAYM6vo27nVhQ
        FM8hQsy3Fo+mYtiFE9fpmvljdllNlhnfjfjsjUr9dmPGIfd9q59KYuqbRdvzXeA3
        AUZWYxUXo7UCAwEAAaNjMGEwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB/wQFMAMB
        Af8wHQYDVR0OBBYEFI0pg+cpCjGSEPHfS6Q7SlmckY5mMB8GA1UdIwQYMBaAFI0p
        g+cpCjGSEPHfS6Q7SlmckY5mMA0GCSqGSIb3DQEBCwUAA4IBAQCsNCHYnsCP2oDY
        TCqsWnqcQLCM1UALVYo6xVAxpVMzVdG+3op6bzGY+mgFnrVfwK9O5o4sxvUE8X53
        yB2hxMM2wBuPVpQns6OazK2lQ5ABJUqr72+uKVxl7I7WZdbYXx5Eb4AOLtcGfhDx
        C/9BxuIVbPUFmEikdR1xEKM4rv8PsVQ7RBKHesUQD9qP4DJsZFom8lgH6EbbusA2
        L5ANl4zcJ2ZQYdlrb2XnRUwG0nSYxb3N0x1cyK2MQ0FAGz995pDRHue8V/FAj9Hb
        PqhxO6W2sjDCB8xTfg8yJAhhxRDTUBxDVF69FFrZWeuLmA/qHRWfXjBTGKVjhH/f
        uTDGX/6I
        -----END CERTIFICATE-----
    - name: UPLOAD_IMAGE_SIZE
      value: 1Gi
    - name: CLIENT_NAME
      value: client.upload-server.cdi.kubevirt.io
    image: registry.stage.redhat.io/container-native-virtualization/virt-cdi-uploadserver@sha256:7a6611d2f738c04bcc93f2062c074e8a80a0107be780df46bfe8cb73fc4c2d52
    imagePullPolicy: IfNotPresent
    name: cdi-upload-server
    readinessProbe:
      failureThreshold: 3
      httpGet:
        path: /healthz
        port: 8080
        scheme: HTTP
      initialDelaySeconds: 2
      periodSeconds: 5
      successThreshold: 1
      timeoutSeconds: 1
    resources:
      limits:
        cpu: "0"
        memory: "0"
      requests:
        cpu: "0"
        memory: "0"
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /data
      name: cdi-data-vol
    - mountPath: /scratch
      name: cdi-scratch-vol
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-ctpkk
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  imagePullSecrets:
  - name: default-dockercfg-sfg85
  priority: 0
  restartPolicy: OnFailure
  schedulerName: default-scheduler
  securityContext:
    runAsUser: 0
  serviceAccount: default


apiVersion: cdi.kubevirt.io/v1alpha1
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: cdi-data-vol
    persistentVolumeClaim:
      claimName: dv-name
  - name: cdi-scratch-vol
    persistentVolumeClaim:
      claimName: dv-name-scratch
  - name: default-token-ctpkk
    secret:


apiVersion: cdi.kubevirt.io/v1alpha1
      defaultMode: 420
      secretName: default-token-ctpkk
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2020-04-27T07:46:04Z"
    message: 'error while running "VolumeBinding" filter plugin for pod "cdi-upload-dv-name":
      pod has unbound immediate PersistentVolumeClaims'
    reason: Unschedulable
    status: "False"
    type: PodScheduled
  phase: Pending
  qosClass: BestEffort

-------------------------------------------------------------------------

$ oc get dv dv-name -oyaml
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
  creationTimestamp: "2020-04-27T07:46:04Z"
  generation: 2
  name: dv-name
  namespace: default
  resourceVersion: "2723569"
  selfLink: /apis/cdi.kubevirt.io/v1alpha1/namespaces/default/datavolumes/dv-name
  uid: 8ced0050-0d83-41bb-bed5-b7ae2625eef5
spec:
  pvc:
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 1Gi
  source:
    upload: {}
status:
  phase: Pending

-------------------------------------------------------------------------

$ oc get pvc
NAME              STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS           AGE
dv-name           Pending                                                             11s
dv-name-scratch   Pending                                      hostpath-provisioner   10s

$ oc get cdiconfig config -oyaml
apiVersion: cdi.kubevirt.io/v1alpha1
kind: CDIConfig
metadata:
  creationTimestamp: "2020-04-24T14:09:03Z"
  generation: 13
  labels:
    app: containerized-data-importer
    cdi.kubevirt.io: ""
  name: config
  ownerReferences:
  - apiVersion: cdi.kubevirt.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: CDI
    name: cdi-kubevirt-hyperconverged
    uid: ae8cfeba-350e-4141-a1d8-22fa7f7ab3ab
  resourceVersion: "2383797"
  selfLink: /apis/cdi.kubevirt.io/v1alpha1/cdiconfigs/config
  uid: 5113a12a-25a5-4859-9f6f-42240b0e45e2
spec:
  scratchSpaceStorageClass: hostpath-provisioner
status:
  defaultPodResourceRequirements:
    limits:
      cpu: "0"
      memory: "0"
    requests:
      cpu: "0"
      memory: "0"
  scratchSpaceStorageClass: hostpath-provisioner
  uploadProxyURL: cdi-uploadproxy-openshift-cnv.apps.bm-rhel.oc4

-------------------------------------------------------------------------

$ oc describe pvc dv-name
Name:          dv-name
Namespace:     default
StorageClass:
Status:        Pending
Volume:
Labels:        app=containerized-data-importer
               cdi-controller=dv-name
Annotations:   cdi.kubevirt.io/storage.pod.phase: Pending
               cdi.kubevirt.io/storage.pod.ready: false
               cdi.kubevirt.io/storage.upload.target:
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Mounted By:    cdi-upload-dv-name
Events:
  Type    Reason         Age                From                         Message
  ----    ------         ----               ----                         -------
  Normal  FailedBinding  14s (x7 over 80s)  persistentvolume-controller  no persistent volumes available for this claim and no storage class is set

-------------------------------------------------------------------------

$ oc describe pvc dv-name-scratch
Name:          dv-name-scratch
Namespace:     default
StorageClass:  hostpath-provisioner
Status:        Pending
Volume:
Labels:        app=containerized-data-importer
               cdi-controller=cdi-upload-dv-name
               cdi.kubevirt.io/storage.import.importPvcName=dv-name
Annotations:   <none>
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Mounted By:    cdi-upload-dv-name
Events:
  Type    Reason                Age               From                         Message
  ----    ------                ----              ----                         -------
  Normal  WaitForFirstConsumer  2s (x8 over 98s)  persistentvolume-controller  waiting for first consumer to be created before binding

-------------------------------------------------------------------------

$ oc get nodes
NAME                                      STATUS   ROLES    AGE     VERSION
cnv-qe-01.cnvqe.lab.eng.rdu2.redhat.com   Ready    master   2d22h   v1.17.1
cnv-qe-02.cnvqe.lab.eng.rdu2.redhat.com   Ready    master   2d22h   v1.17.1
cnv-qe-03.cnvqe.lab.eng.rdu2.redhat.com   Ready    master   2d22h   v1.17.1
cnv-qe-04.cnvqe.lab.eng.rdu2.redhat.com   Ready    worker   2d19h   v1.17.1
cnv-qe-05.cnvqe.lab.eng.rdu2.redhat.com   Ready    worker   2d20h   v1.17.1
cnv-qe-06.cnvqe.lab.eng.rdu2.redhat.com   Ready    worker   2d20h   v1.17.1

Note: 
I'm having difficulty determining if it's a regression issue because we had similar scenarios, but not a scenario where we have only one storage class and it's not defined as default.

Comment 1 Nelly Credi 2020-04-27 10:26:22 UTC

I do not consider this as a blocker, 
but I do think we should add it as a known issue in 2.3 docs
I have added suggeted doc
@Adam, @Natalie please review

Comment 2 Natalie Gavrielov 2020-04-27 12:13:22 UTC

I think this issue is not hpp specific.
I assume that it will probably happen regardless of any storage type used.
The main issue here is that scratchSpaceStorageClass configuration is being ignored and not having default storage class defined reveals it.

Comment 3 Adam Litke 2020-04-27 12:50:42 UTC

Agreed this looks like an issue with CDI and would therefore be storage agnostic.  I agree that it should be documented for 2.3.

Comment 4 Adam Litke 2020-05-06 12:33:38 UTC

Alexander, please take a look.

Comment 5 Alexander Wels 2020-05-06 12:44:36 UTC

So this is not a bug, here is what happened:

1. You ran: virtctl image-upload dv dv-name --size=1Gi --image-path=cirros-0.4.0-x86_64-disk.qcow2 --insecure
2. You have no default storage class.
3. The PVCs are created as expected, and in the yaml of the PVCs you can see that the scratch space PVC is using the hostpath-provisioner storage class, but the target PVC has no storage class, and no storage class is defined.
4. The POD cannot run, because the target PVC cannot be bound. The scratch space PVC is not bound because of WaitForFirstConsumer waiting for a consumer (which would normally be the upload pod, but it cannot run because the target PVC cannot be bound due to no storage class).

In short it is doing exactly what it is supposed to do. If you specify the storage class in the virtcl image-upload command to be hostpath-provisioner, then this will all work.

Please close after verifying that adding the storage class to the target PVC in the upload command works as expected.

Comment 6 Natalie Gavrielov 2020-05-20 11:23:47 UTC

Alexander is right, I was probably too tired when opening this one :), sorry for the hassle.