Bug 2168165 - [4.13]preallocation is always applied when importing image to block storage
Summary: [4.13]preallocation is always applied when importing image to block storage
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: Storage
Version: 4.12.0
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
: 4.13.0
Assignee: Álvaro Romero
QA Contact: Yan Du
URL:
Whiteboard:
Depends On:
Blocks: 2189455 2189460 2189463
TreeView+ depends on / blocked
 
Reported: 2023-02-08 10:56 UTC by Manju
Modified: 2023-07-05 02:21 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2189455 2189460 2189463 (view as bug list)
Environment:
Last Closed: 2023-05-18 02:57:30 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github kubevirt containerized-data-importer pull 2588 0 None Merged Stop forcing preallocation on block imports 2023-02-21 09:04:28 UTC
Github kubevirt containerized-data-importer pull 2654 0 None Merged [release-v1.55] Stop forcing preallocation on block imports 2023-04-04 13:43:11 UTC
Github kubevirt containerized-data-importer pull 2655 0 None Merged [release-v1.49] Stop forcing preallocation on block imports 2023-04-04 13:43:11 UTC
Github kubevirt containerized-data-importer pull 2657 0 None Merged [release-v1.43] Stop forcing preallocation on block imports 2023-04-04 13:43:11 UTC
Red Hat Issue Tracker CNV-25163 0 None None None 2023-02-08 10:59:42 UTC
Red Hat Knowledge Base (Solution) 7005389 0 None None None 2023-04-05 08:25:25 UTC
Red Hat Product Errata RHSA-2023:3205 0 None None None 2023-05-18 02:57:47 UTC

Description Manju 2023-02-08 10:56:44 UTC
Description of problem:

When we create a datavolume by importing an image via http url, preallocation is being applied .We tried setting preallocation to false, but still preallocation is being applied.

storage class name : ocs-external-storagecluster-ceph-rbd

Version-Release number of selected component (if applicable):
Server Version: 4.12.1
OpenShift Virtualization   4.12.0

How reproducible:
100%

Steps to Reproduce:

1.Used below image to create our volume, actual size of this image is 747MB

#qemu-img info rhel-8.5-update-2-x86_64-kvm.qcow2
image: rhel-8.5-update-2-x86_64-kvm.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 747 MiB
cluster_size: 65536
Format specific information:
    compat: 0.10
    refcount bits: 16


2. Created datavol.yaml with preallocation: false 

  # cat datavol.yaml 
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  name: preallocation-false2
spec:
  preallocation: false
  storage:
    resources:
      requests:
        storage: 20Gi
    storageClassName: ocs-external-storagecluster-ceph-rbd
    volumeMode: Block
  source:
    http:
       url: http://10.74.128.145/rhel-8.5-update-2-x86_64-kvm.qcow2


2. Created a dv from the above yaml file

# oc apply -f datavol.yaml 
datavolume.cdi.kubevirt.io/preallocation-false2 created


#oc logs -f importer-preallocation-false2
I0208 10:06:15.478140       1 importer.go:104] Starting importer
I0208 10:06:15.479217       1 importer.go:171] begin import process
I0208 10:06:15.481932       1 data-processor.go:379] Calculating available size
I0208 10:06:15.483183       1 data-processor.go:387] Checking out block volume size.
I0208 10:06:15.483201       1 data-processor.go:399] Request image size not empty.
I0208 10:06:15.483210       1 data-processor.go:404] Target size 21474836480.
I0208 10:06:15.483618       1 nbdkit.go:303] Waiting for nbdkit PID.
I0208 10:06:15.990529       1 nbdkit.go:324] nbdkit ready.
I0208 10:06:15.990584       1 data-processor.go:282] New phase: Convert
I0208 10:06:15.990597       1 data-processor.go:288] Validating image
I0208 10:06:16.165295       1 qemu.go:332] Adding preallocation method: [-o preallocation=falloc]
I0208 10:06:16.170468       1 qemu.go:259] 0.00

The used size of the image is equal to the virtual size of the image.

# rbd du csi-vol-35cc672d-a798-11ed-8a89-0a580a80001a
NAME                                          PROVISIONED  USED  
csi-vol-35cc672d-a798-11ed-8a89-0a580a80001a       20 GiB  10 GiB


Actual results: 

On checking the pod logs for the created volume, we could see that preallocation is being applied and it consumed 10GB from the CEPH end (logs provided in the above step) 



Expected results: 
Since we applied preallocation to false in the datavol.yaml file , the created volume size should consume only 747MiB which is actually the disk size of the img we are using.

Comment 1 nijin ashok 2023-02-10 14:56:59 UTC
The preallocation was enabled always in the block as per [1] (to fix [2] but don't have much info about the root cause).

I tried reproducing the issue with the ubuntu image mentioned in the above issue (after converting to RAW) by using a ceph rbd volume mapped via krbd. I was not able to reproduce the issue with the same "qemu-img convert" options. Not sure if this was because of some bug in qemu-img which was already fixed.

Also, refer to https://github.com/kubevirt/containerized-data-importer/issues/2160.

  
[1] https://github.com/kubevirt/containerized-data-importer/pull/2001
[2] https://github.com/kubevirt/containerized-data-importer/issues/1980

Comment 2 Álvaro Romero 2023-02-10 17:00:19 UTC
Hey @nashok, thank you for reporting this issue!

As you pointed out, this is a behavior that we deliberately included to account for unexpected behaviors in QEMU, and we shouldn't really consider it a bug. However, I agree it may be a good idea to allow disabling preallocation if specified during DataVolume creation.

Since some other users might depend on the current behavior, I'll do some research and discuss with the team some possible solutions.

Comment 3 Álvaro Romero 2023-02-21 09:04:28 UTC
Hi @nashok! The team agreed that we shouldn't keep this behavior after ensuring the original bug had been fixed externally, so the PR with the fix has just been merged and is attached in the links section.

Comment 4 Yan Du 2023-03-07 10:20:15 UTC
Test on CNV-v4.13.0.rhel9-1689

Create DV with preallocation: false, didn't see "Adding preallocation method: [-o preallocation=falloc]" log in importer pod
and the volume size is actually the disk size of the image we are using

$ qemu-img info rhel-85.raw 
image: rhel-85.raw
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 1.68 GiB

sh-5.1$ rbd du ocs-storagecluster-cephblockpool/csi-vol-03c8ce80-08f2-455a-a63f-ab07ed79625d
NAME                                          PROVISIONED  USED   
csi-vol-03c8ce80-08f2-455a-a63f-ab07ed79625d       20 GiB  1.7 GiB

Comment 8 Dan Kenigsberg 2023-04-24 08:05:14 UTC
Do we have BZs for the backported PRs? We would like to tell customers which precise version has this fixed.

Comment 10 Álvaro Romero 2023-04-26 16:11:12 UTC
(In reply to Dan Kenigsberg from comment #8)
> Do we have BZs for the backported PRs? We would like to tell customers which
> precise version has this fixed.

Hey Dan, posting here the backport BZs:

https://bugzilla.redhat.com/show_bug.cgi?id=2189463 (4.12)
https://bugzilla.redhat.com/show_bug.cgi?id=2189460 (4.11)
https://bugzilla.redhat.com/show_bug.cgi?id=2189455 (4.10)

They've been created after the PRs were merged but hope they are still ok.

Comment 11 errata-xmlrpc 2023-05-18 02:57:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Virtualization 4.13.0 Images security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:3205


Note You need to log in before you can comment on or make changes to this bug.