Bug 1731183 - virtctl image-upload to block does not return status as success.
Summary: virtctl image-upload to block does not return status as success.
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: Storage
Version: 2.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 2.1.0
Assignee: Daniel Erez
QA Contact: Gowrishankar Rajaiyan
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-18 14:34 UTC by Gowrishankar Rajaiyan
Modified: 2019-11-04 15:05 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-04 15:05:05 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Gowrishankar Rajaiyan 2019-07-18 14:34:11 UTC
Description of problem: Image upload is successful even though the status returned is not 'success'.


Version-Release number of selected component (if applicable):
hco-bundle-registry:v2.0.0-36

How reproducible: Always


Steps to Reproduce:
1. Ensure that you have block PV's available.
2. Create a DV for upload and ensure that the PVC is in bound state.
3. Perform 'virtctl image-upload' and ensure that you provide '--no-create' arg.

Actual results: Image upload is successful, however, the status code is incorrect.

$ virtctl image-upload --pvc-name=dv-upload-virtctl --image-path=/tmp/pytest-of-cnv-qe-jenkins/pytest-39/test_successful_upload_with_su7/cirros-0.4.0-x86_64-disk.qcow2.xz --pvc-size=3Gi --insecure --no-create
Using existing PVC local-storage/dv-upload-virtctl
Uploading data to https://cdi-uploadproxy-kubevirt-hyperconverged.apps.working.oc4

 12.13 MiB / 12.13 MiB [=================================================================================== 12.13 MiB / 12.13 MiB [=================================================================================== 12.13 MiB / 12.13 MiB [=================================================================================== 12.13 MiB / 12.13 MiB [=====================================================================] 100.00% 1m0s

Post https://cdi-uploadproxy-kubevirt-hyperconverged.apps.working.oc4/v1alpha1/upload: EOF


Expected results: Should return successful status.


Additional info: Post the image-upload I could successfully perform 'virtctl start' and 'virtctl console' to connect to the VMI.

$ virtctl console test-vm-virtctl
Successfully connected to test-vm-virtctl console. The escape sequence is ^]
login as 'cirros' user. default password: 'gocubsgo'. use 'sudo' for root.
cirros login:

Comment 1 Gowrishankar Rajaiyan 2019-07-18 14:34:47 UTC
Environment:

************************************************************************
*          Check if some pods in not Running state                     *
************************************************************************
No pods in not running status in kubevirt-hyperconverged
************************************************************************
*                          oc version                                  *
************************************************************************
oc version: Client Version: version.Info{Major:"4", Minor:"1+", GitVersion:"v4.1.4", GitCommit:"c9e4f28ff", GitTreeState:"clean", BuildDate:"2019-06-26T23:04:27Z", GoVersion:"go1.12.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.4+c62ce01", GitCommit:"c62ce01", GitTreeState:"clean", BuildDate:"2019-06-27T18:14:14Z", GoVersion:"go1.11.6", Compiler:"gc", Platform:"linux/amd64"}
************************************************************************
*                        kubectl version                               *
************************************************************************
kubectl version: Client Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.4+c9e4f28ff", GitCommit:"c9e4f28ff", GitTreeState:"clean", BuildDate:"2019-06-26T23:04:27Z", GoVersion:"go1.12.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.4+c62ce01", GitCommit:"c62ce01", GitTreeState:"clean", BuildDate:"2019-06-27T18:14:14Z", GoVersion:"go1.11.6", Compiler:"gc", Platform:"linux/amd64"}
************************************************************************
*                        virtctl version                               *
************************************************************************
virtctl version: Client Version: version.Info{GitVersion:"v0.17.4", GitCommit:"adfdb8c07830b99fc79d2fd1d004e862ef70979e", GitTreeState:"clean", BuildDate:"2019-06-27T17:03:10Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{GitVersion:"v0.17.4", GitCommit:"adfdb8c07830b99fc79d2fd1d004e862ef70979e", GitTreeState:"clean", BuildDate:"2019-06-27T14:08:17Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
************************************************************************
*                         HCO image id                                 *
************************************************************************
HCO Image Id: NAME                IMAGE_ID
hco-catalogsource   brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/container-native-virtualization/hco-bundle-registry:v2.0.0-36
************************************************************************

Comment 2 Gowrishankar Rajaiyan 2019-07-18 14:37:41 UTC
dv.yaml
```
---
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
  name: dv-upload-virtctl
spec:
  source:
    upload: {}
  pvc:
    storageClassName: local-blk
    volumeMode: Block
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 3Gi
```

$ oc get sc
NAME                  PROVISIONER                    AGE
local-blk (default)   kubernetes.io/no-provisioner   15d
local-sc              kubernetes.io/no-provisioner   15d



$ oc get cdiconfig -o yaml
apiVersion: v1
items:
- apiVersion: cdi.kubevirt.io/v1alpha1
  kind: CDIConfig
  metadata:
    creationTimestamp: "2019-07-03T12:32:22Z"
    generation: 97
    labels:
      app: containerized-data-importer
      cdi.kubevirt.io: ""
    name: config
    ownerReferences:
    - apiVersion: cdi.kubevirt.io/v1alpha1
      blockOwnerDeletion: true
      controller: true
      kind: CDI
      name: cdi-hyperconverged-cluster
      uid: 92001147-9d8e-11e9-8566-fa163ef0fa9a
    resourceVersion: "10476078"
    selfLink: /apis/cdi.kubevirt.io/v1alpha1/cdiconfigs/config
    uid: 9b0054c4-9d8e-11e9-9237-fa163e41d6f0
  spec:
    scratchSpaceStorageClass: local-sc
  status:
    scratchSpaceStorageClass: local-sc
    uploadProxyURL: cdi-uploadproxy-kubevirt-hyperconverged.apps.working.oc4
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""


cdi-upload pod logs:

$ oc logs cdi-upload-dv-upload-virtctl -f
I0718 11:29:21.209474       1 uploadserver.go:70] Upload destination: /dev/blockDevice
I0718 11:29:21.209768       1 uploadserver.go:72] Running server on 0.0.0.0:8443
2019/07/18 11:30:10 http: TLS handshake error from 10.129.2.30:54962: EOF
I0718 11:30:10.233935       1 data-processor.go:237] Calculating available size
I0718 11:30:10.236907       1 data-processor.go:241] Checking out block volume size.
I0718 11:30:10.239035       1 data-processor.go:249] Request image size not empty.
I0718 11:30:10.239055       1 data-processor.go:254] Target size 3Gi.
I0718 11:30:10.239209       1 data-processor.go:167] New phase: TransferScratch
I0718 11:30:10.243157       1 util.go:140] Writing data...
I0718 11:30:10.682783       1 data-processor.go:167] New phase: Process
I0718 11:30:10.682808       1 data-processor.go:167] New phase: Convert
I0718 11:30:10.682813       1 data-processor.go:173] Validating image
I0718 11:31:20.451688       1 data-processor.go:167] New phase: Resize
I0718 11:31:20.470421       1 data-processor.go:167] New phase: Complete
I0718 11:31:20.470509       1 util.go:37] deleting file: /scratch/tmpimage
I0718 11:31:20.471469       1 uploadserver.go:254] Wrote data to /dev/blockDevice
I0718 11:31:20.471719       1 uploadserver.go:146] Shutting down http server after successful upload
I0718 11:31:20.472031       1 uploadserver.go:80] UploadServer successfully exited

Comment 3 Daniel Erez 2019-08-26 05:44:50 UTC
Verified on virtctl v0.20.2:

$ ./virtctl-v0.20.2-linux-amd64 image-upload --pvc-name=dv-upload-virtctl --image-path=cirros-0.4.0-x86_64-disk.img --pvc-size=3Gi --insecure --no-create
Using existing PVC default/dv-upload-virtctl
Waiting for PVC dv-upload-virtctl upload pod to be running...
Pod now running
Uploading data to https://cdi-uploadproxy-cdi.apps.working.oc4

 12.13 MiB / 12.13 MiB [==================================================================================] 100.00% 0s

Uploading cirros-0.4.0-x86_64-disk.img completed successfully

Comment 4 Qixuan Wang 2019-09-18 08:13:43 UTC
Tested with hco-bundle-registry-v2.1.0-47, CDI v2.1.0-14, virtctl client v0.20.1, virtctl server v0.20.5
Large windows image is uploaded successfully and VM can run with it. The bug has been fixed. Move it to VERIFIED, thanks.

Here is the test result:
1. Prepare a DV and then generate PVC
[cnv-qe-jenkins@cnv-executor-ying ~]$ cat dv.yaml
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
  name: dv-upload-virtctl
spec:
  source:
    upload: {}
  pvc:
    storageClassName: rook-ceph-block
    volumeMode: Block
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 28Gi

[cnv-qe-jenkins@cnv-executor-ying ~]$ oc create -f dv.yaml
datavolume.cdi.kubevirt.io/dv-upload-virtctl created

[cnv-qe-jenkins@cnv-executor-ying ~]$ oc get pvc
NAME                        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
dv-upload-virtctl           Bound    pvc-6d04e2ac-d9ea-11e9-b549-fa163ed3ad0d   28Gi       RWO            rook-ceph-block   21s
dv-upload-virtctl-scratch   Bound    pvc-6d42dbb2-d9ea-11e9-b549-fa163ed3ad0d   28Gi       RWO            rook-ceph-block   20s

2. Upload a local win10 image
[cnv-qe-jenkins@cnv-executor-ying ~]$ virtctl image-upload --storage-class=rook-ceph-block --image-path=./cirros-0.4.0-x86_64-disk.img --pvc-size=28Gi --block-volume --pvc-name=dv-upload-virtctl --insecure --no-create
Using existing PVC bug/dv-upload-virtctl
Uploading data to https://cdi-uploadproxy-openshift-cnv.apps.working.oc4

 12.13 MiB / 12.13 MiB [=================================================================================================================] 100.00% 4s

Uploading ./cirros-0.4.0-x86_64-disk.img completed successfully

[cnv-qe-jenkins@cnv-executor-ying ~]$ oc logs -f cdi-upload-dv-upload-virtctl
I0918 07:55:03.783223       1 uploadserver.go:61] Upload destination: /dev/cdi-block-volume
I0918 07:55:03.783736       1 uploadserver.go:63] Running server on 0.0.0.0:8443
^[[A^[[D^[[D^[[D^[[D^[[D^[[D2019/09/18 07:55:11 http: TLS handshake error from 10.130.0.41:48254: EOF
I0918 07:55:11.310112       1 uploadserver.go:244] Content type header is ""
I0918 07:55:11.310163       1 data-processor.go:252] Calculating available size
I0918 07:55:11.330825       1 data-processor.go:256] Checking out block volume size.
I0918 07:55:11.334349       1 data-processor.go:264] Request image size not empty.
I0918 07:55:11.334413       1 data-processor.go:269] Target size 28Gi.
I0918 07:55:11.335151       1 data-processor.go:182] New phase: TransferScratch
I0918 07:55:11.372494       1 util.go:143] Writing data...
^[[AI0918 07:55:12.741861       1 data-processor.go:182] New phase: Process
I0918 07:55:12.741936       1 data-processor.go:182] New phase: Convert
I0918 07:55:12.741943       1 data-processor.go:188] Validating image
I0918 07:55:15.966951       1 data-processor.go:182] New phase: Resize
I0918 07:55:15.974501       1 data-processor.go:182] New phase: Complete
I0918 07:55:15.974595       1 util.go:37] deleting file: /scratch/tmpimage
I0918 07:55:15.982319       1 uploadserver.go:263] Wrote data to /dev/cdi-block-volume
I0918 07:55:15.982491       1 uploadserver.go:151] Shutting down http server after successful upload
I0918 07:55:16.483506       1 uploadserver.go:71] UploadServer successfully exited

3. Scratch space retires after image upload successfully
[cnv-qe-jenkins@cnv-executor-ying ~]$ oc get pvc
NAME                   STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
dv-upload-virtctl      Bound    pvc-6d04e2ac-d9ea-11e9-b549-fa163ed3ad0d   28Gi       RWO            rook-ceph-block   2m29s

4. Create a VMI with image put on dv-upload-virtctl PVC, VM can run
[cnv-qe-jenkins@cnv-executor-ying ~]$ oc get pod
NAME                            READY   STATUS    RESTARTS   AGE
virt-launcher-win10-vmi-zg5n9   1/1     Running   0          8m4s


Note You need to log in before you can comment on or make changes to this bug.