Description of problem: cdi doesn't work with servers that reject HEAD requests (github links)" Version-Release number of selected component (if applicable): <registry>/container-native-virtualization-hostpath-provisioner-rhel8@sha256:6461956d902351e6c3d08dbd5a6097bae9ca4e52404409f6d2cb8b819f2354e9 <registry>/container-native-virtualization-hyperconverged-cluster-operator@sha256:b449706d20bd97af7915804f493b32d6099c302ffe7a524a2d6cfab8a2658cf1 How reproducible: always Steps to Reproduce: oc create -f file below: apiVersion: cdi.kubevirt.io/v1alpha1 kind: DataVolume metadata: name: hostpath-dv-import spec: source: http: url: "https://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img" pvc: accessModes: - ReadWriteMany resources: requests: storage: 100Mi storageClassName: hostpath-provisioner Actual results: dv is ImportInProgress state hostpath-dv-import ImportInProgress 88m oc logs -f importer-hostpath-dv-import I0223 15:14:38.992270 1 importer.go:51] Starting importer I0223 15:14:38.992661 1 importer.go:107] begin import process E0223 15:14:39.307591 1 http-datasource.go:319] http: expected status code 200, got 403 E0223 15:14:39.307647 1 importer.go:113] expected status code 200, got 403. Status: 403 Forbidden kubevirt.io/containerized-data-importer/pkg/importer.getContentLength /go/src/kubevirt.io/containerized-data-importer/pkg/importer/http-datasource.go:320 kubevirt.io/containerized-data-importer/pkg/importer.createHTTPReader /go/src/kubevirt.io/containerized-data-importer/pkg/importer/http-datasource.go:249 kubevirt.io/containerized-data-importer/pkg/importer.NewHTTPDataSource /go/src/kubevirt.io/containerized-data-importer/pkg/importer/http-datasource.go:80 main.main /go/src/kubevirt.io/containerized-data-importer/cmd/cdi-importer/importer.go:111 runtime.main /usr/lib/golang/src/runtime/proc.go:200 runtime.goexit /usr/lib/golang/src/runtime/asm_amd64.s:1337 Expected results: dv succeeded Additional info:
Another way to get this failure (without CDI): curl -i -L -I https://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img Gets: HTTP/1.1 403 Forbidden
I can make the Go client side in pkg/importer/http-datasource.go use the Content-Length that appears in GET requests. But we later invoke qemu-img on the same URL and it shares this limitation. That is, the failure also happens with: qemu-img convert -t none -p -O raw -o preallocation=falloc https://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img out.img
It works with the cirros server before, so mark it as a regression. I hope this issue gets into 2.3 because it blocks UI testing, and URL cannot be a promise provision source on UI if it just works with some server.
Maya, when you attach the PR into bug, then you can change it to POST status.
Maya, checking your PR already merged, then you can change the bug status to MODIFIED.
Verified with CNV 2.4, this is the importer pod log: 0610 12:15:23.752712 1 importer.go:51] Starting importer I0610 12:15:23.752835 1 importer.go:108] begin import process E0610 12:15:24.254972 1 http-datasource.go:328] http: expected status code 200, got 403 I0610 12:15:24.424980 1 data-processor.go:275] Calculating available size I0610 12:15:24.426559 1 data-processor.go:283] Checking out file system volume size. I0610 12:15:24.426595 1 data-processor.go:287] Request image size not empty. I0610 12:15:24.426627 1 data-processor.go:292] Target size 1Gi. I0610 12:15:24.441337 1 data-processor.go:205] New phase: TransferScratch I0610 12:15:24.445743 1 util.go:144] Writing data... I0610 12:15:30.629943 1 data-processor.go:205] New phase: Process I0610 12:15:30.629983 1 data-processor.go:205] New phase: Convert I0610 12:15:30.629988 1 data-processor.go:211] Validating image I0610 12:15:31.764056 1 data-processor.go:205] New phase: Resize I0610 12:15:31.782292 1 data-processor.go:268] Expanding image size to: 1Gi I0610 12:15:31.798263 1 data-processor.go:205] New phase: Complete I0610 12:15:31.798355 1 util.go:37] deleting file: /scratch/tmpimage I0610 12:15:31.800396 1 importer.go:171] Import complete
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:3194