Bug 1753654
| Summary: | clone fails when using filesystem volumeMode on ceph rbd backend | ||
|---|---|---|---|
| Product: | Container Native Virtualization (CNV) | Reporter: | Daniel Erez <derez> |
| Component: | Storage | Assignee: | Alexander Wels <awels> |
| Status: | CLOSED ERRATA | QA Contact: | Kevin Alon Goldblatt <kgoldbla> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 2.1.0 | CC: | alitke, awels, cnv-qe-bugs, frolland, fsimonce, ncredi, ngavrilo, sgordon |
| Target Milestone: | --- | ||
| Target Release: | 2.1.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | hco-bundle-registry-container-v2.1.0-69 -- cdi-2.1.0-17 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-10-31 14:08:47 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
It looks like a file system overhead . (XFS) As a workaround, by increasing the size of the target PVC by 1Mi the clone will succeed. @Alexander, check v1.10.6. It seems you forgot to backport https://github.com/kubevirt/containerized-data-importer/pull/966 to release-v1.10. Can you check it please? We release v1.10.6 before 996 was merged, next release will contain the PR. Verified with the following code:
----------------------------------------
v2.1.0-latest
virt-cdi-apiserver:v2.1.0-20
Verified with the following scenario:
----------------------------------------
1. Create dv on sc rook-ceph-block(file)
2. Create clone of dv >>>>> created successfully
dv yaml;
-----------------
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
name: source-dv
spec:
pvc:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 500Mi
source:
http:
url: "https://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img"
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
annotations:
name: target-dv
spec:
pvc:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 500Mi
source:
pvc:
name: source-dv
namespace: default
Moving to VERIFIED!
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:3282 |
Description of problem: The cloner tests are failing due to insufficient space in the target, see [logs] of "Tests Suite.[rfe_id:1277][crit:high][vendor:cnv-qe][level:component]Cloner Test Suite [test_id:1354]Should clone data within same namespace" Version-Release number of selected component (if applicable): 2.1.0 How reproducible: 100% Steps to Reproduce: 1. Create source DataVolume - see [source dv yaml] 2. Create target DataVolume - see [target dv yaml] Actual results: Fails to clone. Expected results: Should succeed. Additional info: [logs] source: 09:41:38 Logs for container cdi-clone-source 09:41:38 VOLUME_MODE=filesystem 09:41:38 MOUNT_POINT=/var/run/cdi/clone/source 09:41:38 /var/run/cdi/clone/source / 09:41:38 UPLOAD_BYTES=495091734 09:41:38 ./ 09:41:38 ./disk.img 09:41:38 I0919 06:41:02.647733 11 clone-source.go:106] content_type is "filesystem-clone" 09:41:38 I0919 06:41:02.648420 11 clone-source.go:107] upload_bytes is 495091734 09:41:38 I0919 06:41:02.648435 11 clone-source.go:117] Starting cloner target 09:41:38 I0919 06:41:03.346326 11 clone-source.go:129] Set header to filesystem-clone 09:41:38 I0919 06:41:03.649783 11 prometheus.go:67] 0.57 09:41:38 I0919 06:41:04.649937 11 prometheus.go:67] 3.40 09:41:38 I0919 06:41:05.650132 11 prometheus.go:67] 30.27 09:41:38 I0919 06:41:06.650325 11 prometheus.go:67] 58.49 09:41:38 I0919 06:41:07.650629 11 prometheus.go:67] 87.04 09:41:38 I0919 06:41:08.083239 11 clone-source.go:96] Wrote 495093760 bytes 09:41:38 I0919 06:41:08.653213 11 prometheus.go:67] 100.00 09:41:38 F0919 06:41:19.711272 11 clone-source.go:138] Unexpected status code 500 09:41:38 goroutine 1 [running]: 09:41:38 kubevirt.io/containerized-data-importer/vendor/k8s.io/klog.stacks(0xc0002bbd00, 0xc0002ec000, 0x4e, 0xa3) 09:41:38 /go/src/kubevirt.io/containerized-data-importer/vendor/k8s.io/klog/klog.go:828 +0xd4 09:41:38 kubevirt.io/containerized-data-importer/vendor/k8s.io/klog.(*loggingT).output(0x1a999e0, 0xc000000003, 0xc00018c2c0, 0x1a327ef, 0xf, 0x8a, 0x0) 09:41:38 /go/src/kubevirt.io/containerized-data-importer/vendor/k8s.io/klog/klog.go:779 +0x306 09:41:38 kubevirt.io/containerized-data-importer/vendor/k8s.io/klog.(*loggingT).printf(0x1a999e0, 0xc000000003, 0x1052f41, 0x19, 0xc000521f08, 0x1, 0x1) 09:41:38 /go/src/kubevirt.io/containerized-data-importer/vendor/k8s.io/klog/klog.go:678 +0x14b 09:41:38 kubevirt.io/containerized-data-importer/vendor/k8s.io/klog.Fatalf(0x1052f41, 0x19, 0xc000521f08, 0x1, 0x1) 09:41:38 /go/src/kubevirt.io/containerized-data-importer/vendor/k8s.io/klog/klog.go:1207 +0x67 09:41:38 main.main() 09:41:38 /go/src/kubevirt.io/containerized-data-importer/cmd/cdi-cloner/clone-source.go:138 +0x88b 09:41:38 target: 09:47:37 Logs for container cdi-upload-server 09:47:37 I0919 06:46:04.256760 1 uploadserver.go:61] Upload destination: /data/disk.img 09:47:37 I0919 06:46:04.257267 1 uploadserver.go:63] Running server on 0.0.0.0:8443 09:47:37 I0919 06:46:45.648267 1 uploadserver.go:244] Content type header is "filesystem-clone" 09:47:37 I0919 06:46:45.648845 1 util.go:156] begin untar to /data... 09:47:37 I0919 06:47:05.118230 1 util.go:175] /usr/bin/tar: ./disk.img: Wrote only 1024 of 10240 bytes 09:47:37 /usr/bin/tar: Exiting with failure status due to previous errors 09:47:37 09:47:37 E0919 06:47:05.118361 1 util.go:176] exit status 2 09:47:37 E0919 06:47:05.119188 1 uploadserver.go:252] Saving stream failed: error unarchiving to /data: exit status 2 09:47:37 I0919 06:47:07.029362 1 uploadserver.go:244] Content type header is "filesystem-clone" 09:47:37 I0919 06:47:07.029530 1 util.go:37] deleting file: /data/disk.img 09:47:37 I0919 06:47:07.073778 1 util.go:156] begin untar to /data... [source dv yaml] apiVersion: cdi.kubevirt.io/v1alpha1 kind: DataVolume metadata: name: source-dv spec: pvc: accessModes: - ReadWriteOnce resources: requests: storage: 500Mi source: http: url: http://cdi-file-host.openshift-cnv/tinyCore.iso [target dv yaml] apiVersion: cdi.kubevirt.io/v1alpha1 kind: DataVolume metadata: annotations: name: target-dv spec: pvc: accessModes: - ReadWriteOnce resources: requests: storage: 500Mi source: pvc: name: source-dv namespace: default