Bug 1868690

Patch posted:
https://www.redhat.com/archives/libguestfs/2020-September/msg00004.html

Upstream in:
https://github.com/libguestfs/virt-v2v/commit/9a5974fa3bc038e5e5dbb9605a6db77d06e7bf77

mxie, could you ACK this please?

-os /dev is pretty dangerous!!  You can make it safer by making
a directory, eg:

# mkdir /tmp/output
# mknod /tmp/output/sda3 b 8 3
# virt-v2v ... -os /tmp/output

I think even symbolic links will work there too.

---

In step 1.2, the input guest (esx6.7-rhel8.3-x86_64) has size
12884901888 bytes which is > 10 GB.  qemu-img correctly prints
the error message:

> qemu-img: output file is smaller than input file

In step 2.2, it's basically the same as above.

But in step 2.3 you say:

> 2.3 Check the  block device after v2v conversion, found the block device is overwrote
> # ls -l /dev/sda3
> brw-rw----. 1 root disk 8, 3 Apr 29 21:39 /dev/sda3

However this looks fine to me?  It's still a block device, with
the same major and minor.  The timestamp has changed, but I'm
not sure if that is a problem.

Thanks Richard, the block device won't be overwrote(timestamp isn't changed) if symbolic the partition to a directory, so the bug has been fixed.

# mknod /home/partition/sda3 b 8 3 

# qemu-img info /home/partition/sda3
image: /home/partition/sda3
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 0 B

# ls -l /home/partition/sda3
brw-r--r--. 1 root root 8, 3 Apr 29 22:48 /home/partition/sda3

# virt-v2v  -ic esx://root.75.219/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk7.0 -io vddk-thumbprint=A5:A8:BB:A2:33:10:3E:37:C0:A4:BA:BF:99:F5:87:D1:A4:F4:38:F6  esx6.7-rhel8.3-x86_64 -ip /home/esxpw -o json -os /home/partition -oo 'json-disks-pattern=sda3'
[   0.0] Opening the source -i libvirt -ic esx://root.75.219/?no_verify=1 esx6.7-rhel8.3-x86_64 -it vddk  -io vddk-libdir=/home/vddk7.0 -io vddk-thumbprint=A5:A8:BB:A2:33:10:3E:37:C0:A4:BA:BF:99:F5:87:D1:A4:F4:38:F6
[   1.4] Creating an overlay to protect the source from being modified
[   2.0] Opening the overlay
[   6.3] Inspecting the overlay
[  18.2] Checking for sufficient free disk space in the guest
[  18.2] Estimating space required on target for each disk
[  18.2] Converting Red Hat Enterprise Linux 8.3 (Ootpa) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[  65.7] Mapping filesystem data to avoid copying unused and blank areas
[  66.4] Closing the overlay
[  66.5] Assigning disks to buses
[  66.5] Checking if the guest needs BIOS or UEFI to boot
[  66.5] Initializing the target -o json -os /home/partition
[  66.5] Copying disk 1/1 to /home/partition/sda3 (raw)
qemu-img: output file is smaller than input file

virt-v2v: error: qemu-img command failed, see earlier errors

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]

# ls -l /home/partition/sda3
brw-r--r--. 1 root root 8, 3 Apr 29 22:48 /home/partition/sda3

Verify the bug with below builds:
virt-v2v-1.42.0-12.module+el8.5.0+10976+d40a93e9.x86_64
libguestfs-1.44.0-3.module+el8.5.0+10681+17a9b157.x86_64
libvirt-client-7.0.0-13.module+el8.4.0+10604+5608c2b4.x86_64
qemu-kvm-6.0.0-16.module+el8.5.0+10848+2dccc46d.x86_64
nbdkit-1.24.0-1.module+el8.4.0+9341+96cf2672.x86_64

Steps:
Scenario1: Use ceph image as block device:

1.1 Map a ceph storage as block device on v2v server
# # qemu-img info rbd:libvirt-pool/mxie:id=admin:key=AQBm9fldc9zhMhAAeDDedFhu55XxxxjV1YhdqDOkQ==:auth_supported=cephx:mon_host=10.66.xx.xx
image: json:{"driver": "raw", "file": {"pool": "libvirt-pool", "image": "mxie", "driver": "rbd", "namespace": "", "user": "admin"}}
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: unavailable
cluster_size: 4194304

# rbd map mxie --pool libvirt-pool 
/dev/rbd0

# mknod /home/ceph-block-disk/ceph-disk b 252 0

# qemu-img info /home/ceph-block-disk/ceph-disk
image: /home/ceph-block-disk/ceph-disk
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 0 B

# ls -l /home/ceph-block-disk/ceph-disk
brw-r--r--. 1 root root 252, 0 Apr 29 21:29 /home/ceph-block-disk/ceph-disk

1.2 Convert a guest from VMware to local block device by virt-v2v, make sure disk size of guest is greater than the size of the block device and converted disk name is same with the block disk
# virt-v2v  -ic esx://root.75.219/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk7.0 -io vddk-thumbprint=A5:A8:BB:A2:33:10:3E:37:C0:A4:BA:BF:99:F5:87:D1:A4:F4:38:F6  esx6.7-rhel8.3-x86_64 -ip /home/esxpw -o json -os /home/ceph-block-disk -oo 'json-disks-pattern=ceph-disk'
[   0.0] Opening the source -i libvirt -ic esx://root.75.219/?no_verify=1 esx6.7-rhel8.3-x86_64 -it vddk  -io vddk-libdir=/home/vddk7.0 -io vddk-thumbprint=A5:A8:BB:A2:33:10:3E:37:C0:A4:BA:BF:99:F5:87:D1:A4:F4:38:F6
[   1.3] Creating an overlay to protect the source from being modified
[   2.2] Opening the overlay
[   7.7] Inspecting the overlay
[  22.2] Checking for sufficient free disk space in the guest
[  22.2] Estimating space required on target for each disk
[  22.2] Converting Red Hat Enterprise Linux 8.3 (Ootpa) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[  92.3] Mapping filesystem data to avoid copying unused and blank areas
[  93.4] Closing the overlay
[  93.7] Assigning disks to buses
[  93.7] Checking if the guest needs BIOS or UEFI to boot
[  93.7] Initializing the target -o json -os /home/ceph-block-disk
[  93.7] Copying disk 1/1 to /home/ceph-block-disk/ceph-disk (raw)
qemu-img: output file is smaller than input file

virt-v2v: error: qemu-img command failed, see earlier errors

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]


1.3 Check the block device after v2v conversion, found ceph block device is not overwrote.
# ls -l /home/ceph-block-disk/ceph-disk
brw-r--r--. 1 root root 252, 0 Apr 29 21:29 /home/ceph-block-disk/ceph-disk

# qemu-img info /home/ceph-block-disk/ceph-disk
image: /home/ceph-block-disk/ceph-disk
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 0 B


1.4  Convert a guest from Xen to local block device by virt-v2v, make sure the disk size of guest is smaller than the size of the block device and converted disk name is same with the block disk

# qemu-img info /var/lib/xen/images/xen-hvm-rhel6.10-x86_64.img 
image: /var/lib/xen/images/xen-hvm-rhel6.10-x86_64.img
file format: raw
virtual size: 7.8G (8388608000 bytes)
disk size: 7.8G

# virt-v2v -ic xen+ssh://root.224.33 xen-hvm-rhel6.10-x86_64 -of raw -o json -os /home/ceph-block-disk -oo 'json-disks-pattern=ceph-disk'
[   0.0] Opening the source -i libvirt -ic xen+ssh://root.224.33 xen-hvm-rhel6.10-x86_64
[   1.5] Creating an overlay to protect the source from being modified
qemu-img: warning: Deprecated use of backing file without explicit backing format (detected format of raw)
[   1.9] Opening the overlay
[   6.6] Inspecting the overlay
[  19.8] Checking for sufficient free disk space in the guest
[  19.8] Estimating space required on target for each disk
[  19.8] Converting Red Hat Enterprise Linux Server release 6.10 (Santiago) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 106.6] Mapping filesystem data to avoid copying unused and blank areas
[ 107.3] Closing the overlay
[ 107.3] Assigning disks to buses
[ 107.3] Checking if the guest needs BIOS or UEFI to boot
[ 107.3] Initializing the target -o json -os /home/ceph-block-disk
[ 107.3] Copying disk 1/1 to /home/ceph-block-disk/ceph-disk (raw)
    (100.00/100%)
[ 485.4] Creating output metadata
[ 485.4] Finishing off


1.5 Check the block device after v2v conversion, found ceph block device is overwrote and size of block device is not changed

# ls -l /home/ceph-block-disk/ceph-disk
brw-r--r--. 1 root root 252, 0 May 14 17:14 /home/ceph-block-disk/ceph-disk

# qemu-img info /home/ceph-block-disk/ceph-disk
image: /home/ceph-block-disk/ceph-disk
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 0 B


Scenario2: Make local partition as block device

2.1 Prepare a independent partition as block device
# qemu-img info /dev/sda3
image: /dev/sda3
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 0 B

# mknod /home/partition/sda3 b 8 3 

# qemu-img info /home/partition/sda3
image: /home/partition/sda3
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 0 B

# ls -l /home/partition/sda3
brw-r--r--. 1 root root 8, 3 Apr 29 23:08 /home/partition/sda3

2.2 Convert a guest from VMware to local block device by virt-v2v, make sure disk size of guest is greater than the size of the block device and converted disk name is same with the block disk
# virt-v2v  -ic esx://root.75.219/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk7.0 -io vddk-thumbprint=A5:A8:BB:A2:33:10:3E:37:C0:A4:BA:BF:99:F5:87:D1:A4:F4:38:F6  esx6.7-rhel8.3-x86_64 -ip /home/esxpw -o json -os /home/partition -oo 'json-disks-pattern=sda3'
[   0.0] Opening the source -i libvirt -ic esx://root.75.219/?no_verify=1 esx6.7-rhel8.3-x86_64 -it vddk  -io vddk-libdir=/home/vddk7.0 -io vddk-thumbprint=A5:A8:BB:A2:33:10:3E:37:C0:A4:BA:BF:99:F5:87:D1:A4:F4:38:F6
[   1.4] Creating an overlay to protect the source from being modified
[   2.3] Opening the overlay
[   6.6] Inspecting the overlay
[  18.6] Checking for sufficient free disk space in the guest
[  18.6] Estimating space required on target for each disk
[  18.6] Converting Red Hat Enterprise Linux 8.3 (Ootpa) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[  69.9] Mapping filesystem data to avoid copying unused and blank areas
[  70.7] Closing the overlay
[  70.8] Assigning disks to buses
[  70.8] Checking if the guest needs BIOS or UEFI to boot
[  70.8] Initializing the target -o json -os /home/partition
[  70.8] Copying disk 1/1 to /home/partition/sda3 (raw)
qemu-img: output file is smaller than input file

virt-v2v: error: qemu-img command failed, see earlier errors

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]

2.3 Check the block device after v2v conversion, found the block device is not overwrote
# ls -l /home/partition/sda3
brw-r--r--. 1 root root 8, 3 Apr 29 23:08 /home/partition/sda3

2.4 Convert a guest from Xen to local block device by virt-v2v, make sure the disk size of guest is smaller than the size of the block device and converted disk name is same with the block disk
# virt-v2v -ic xen+ssh://root.224.33 xen-hvm-rhel6.10-x86_64 -of raw -o json -os /home/partition -oo 'json-disks-pattern=sda3'
[   0.0] Opening the source -i libvirt -ic xen+ssh://root.224.33 xen-hvm-rhel6.10-x86_64
[   3.2] Creating an overlay to protect the source from being modified
qemu-img: warning: Deprecated use of backing file without explicit backing format (detected format of raw)
[   3.6] Opening the overlay
[   8.3] Inspecting the overlay
[  21.4] Checking for sufficient free disk space in the guest
[  21.4] Estimating space required on target for each disk
[  21.4] Converting Red Hat Enterprise Linux Server release 6.10 (Santiago) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 107.9] Mapping filesystem data to avoid copying unused and blank areas
[ 108.5] Closing the overlay
[ 108.6] Assigning disks to buses
[ 108.6] Checking if the guest needs BIOS or UEFI to boot
[ 108.6] Initializing the target -o json -os /home/partition
[ 108.6] Copying disk 1/1 to /home/partition/sda3 (raw)
    (100.00/100%)
[ 401.9] Creating output metadata
[ 401.9] Finishing off


2.5 Check the  block device after v2v conversion, found ceph block device is overwrote
# ls -l /home/partition/sda3
brw-r--r--. 1 root root 8, 3 May 14 18:48 /home/partition/sda3

Result:
    Bug has been fixed according to above test results, move the bug from ON_QA to VERIFIED

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:av bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4684