Bug 1913577 - Windows guest has no disk after v2v converting to rhv4.4 if convert guest via rhv-upload and not install drivers for guest from virtio-win during conversion
Summary: Windows guest has no disk after v2v converting to rhv4.4 if convert guest via...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: virt-v2v
Version: 8.4
Hardware: x86_64
OS: Unspecified
medium
medium
Target Milestone: rc
: 8.4
Assignee: Richard W.M. Jones
QA Contact: tingting zheng
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-01-07 07:32 UTC by mxie@redhat.com
Modified: 2021-06-16 11:15 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-06-16 10:22:19 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
windows-guest-has-no-disk-after-v2v-conversion.png (78.50 KB, image/png)
2021-01-07 07:32 UTC, mxie@redhat.com
no flags Details
vdsm.log (681.67 KB, text/plain)
2021-01-07 07:35 UTC, mxie@redhat.com
no flags Details
engine.log (257.98 KB, text/plain)
2021-01-07 07:37 UTC, mxie@redhat.com
no flags Details
virt-v2v-convert-windows-guest-rhv-upload-without-virtio-win.log (527.87 KB, text/plain)
2021-01-07 07:38 UTC, mxie@redhat.com
no flags Details

Description mxie@redhat.com 2021-01-07 07:32:28 UTC
Created attachment 1745202 [details]
windows-guest-has-no-disk-after-v2v-conversion.png

Description of problem:
Windows guest has no disk after v2v converting to rhv4.4 if convert guest via rhv-upload and not install drivers for guest from virtio-win during conversion

Version-Release number of selected component (if applicable):
virt-v2v-1.42.0-6.module+el8.4.0+8855+a9e237a9.x86_64
libguestfs-1.42.0-2.module+el8.4.0+8855+a9e237a9.x86_64
libvirt-client-6.10.0-1.module+el8.4.0+8898+a84e86e1.x86_64
qemu-kvm-5.2.0-2.module+el8.4.0+9186+ec44380f.x86_64
nbdkit-1.22.0-2.module+el8.4.0+8855+a9e237a9.x86_64


How reproducible:
100%

Steps to Reproduce:
1.Remove virtio-win package on v2v conversion server
# rpm -q virtio-win
package virtio-win is not installed

2.Convert a windows guest from VMware to rhv4.4 via rhv-upload by virt-v2v, the conversion can finish without error
# virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1  esx7.0-win2019-x86_64 -b ovirtmgmt -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78 -o rhv-upload -of qcow2 -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -op /home/rhvpasswd  -os nfs_data -n ovirtmgmt -ip /home/passwd 
[   1.0] Opening the source -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 esx7.0-win2019-x86_64 -it vddk  -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78
[   2.8] Creating an overlay to protect the source from being modified
[   3.7] Opening the overlay
[  11.0] Inspecting the overlay
[  16.3] Checking for sufficient free disk space in the guest
[  16.3] Estimating space required on target for each disk
[  16.3] Converting Windows Server 2019 Standard to run on KVM
virt-v2v: warning: /usr/share/virt-tools/pnp_wait.exe is missing.  
Firstboot scripts may conflict with PnP.
virt-v2v: warning: QEMU Guest Agent MSI not found on tools ISO/directory. 
You may want to install the guest agent manually after conversion.
virt-v2v: warning: there are no virtio drivers available for this version 
of Windows (10.0 x86_64 Server).  virt-v2v looks for drivers in 
/usr/share/virtio-win

The guest will be configured to use slower emulated devices.
virt-v2v: This guest does not have virtio drivers installed.
[  20.0] Mapping filesystem data to avoid copying unused and blank areas
[  21.1] Closing the overlay
[  21.4] Assigning disks to buses
[  21.4] Checking if the guest needs BIOS or UEFI to boot
[  21.4] Initializing the target -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -op /home/rhvpasswd -os nfs_data
[  22.9] Copying disk 1/1 to qemu URI json:{ "file.driver": "nbd", "file.path": "/tmp/v2vnbdkit.Kewnz3/nbdkit4.sock", "file.export": "/" } (qcow2)
    (100.00/100%)
[ 891.9] Creating output metadata
[ 893.9] Finishing off


3.Check guest on rhv4.4, but find guest has no disk, pls refer to screenshot"windows-guest-has-no-disk-after-v2v-conversion.png"

4.Find error info "VM esx7.0-win2019-x86_64 has been imported from the given configuration but the following disk(s) failed to attach: 19ed4885-90b3-42e0-9df8-3a022fccc3ce." in engine.log


Actual result:
As above description

Expected results:
Windows guest has disk after v2v converting to rhv4.4 if convert guest via rhv-upload and not install drivers for guest from virtio-win during v2v conversion


Additional info:
1. Can't reproduce the problem when convert a windows guest to rhv4.3 via rhv-upload and not install drivers for guest from virtio-win during conversion

1.1 # virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1  esx7.0-win2019-x86_64 -b ovirtmgmt -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78 -o rhv-upload -of qcow2 -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /home/rhvpasswd  -os nfs_data -n ovirtmgmt -ip /home/passwd 
[   0.9] Opening the source -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 esx7.0-win2019-x86_64 -it vddk  -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78
[   2.6] Creating an overlay to protect the source from being modified
[   3.5] Opening the overlay
[  10.7] Inspecting the overlay
[  15.9] Checking for sufficient free disk space in the guest
[  15.9] Estimating space required on target for each disk
[  15.9] Converting Windows Server 2019 Standard to run on KVM
virt-v2v: warning: /usr/share/virt-tools/pnp_wait.exe is missing.  
Firstboot scripts may conflict with PnP.
virt-v2v: warning: QEMU Guest Agent MSI not found on tools ISO/directory. 
You may want to install the guest agent manually after conversion.
virt-v2v: warning: there are no virtio drivers available for this version 
of Windows (10.0 x86_64 Server).  virt-v2v looks for drivers in 
/usr/share/virtio-win

The guest will be configured to use slower emulated devices.
virt-v2v: This guest does not have virtio drivers installed.
[  19.3] Mapping filesystem data to avoid copying unused and blank areas
[  20.4] Closing the overlay
[  20.7] Assigning disks to buses
[  20.7] Checking if the guest needs BIOS or UEFI to boot
[  20.7] Initializing the target -o rhv-upload -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /home/rhvpasswd -os nfs_data
[  22.1] Copying disk 1/1 to qemu URI json:{ "file.driver": "nbd", "file.path": "/tmp/v2vnbdkit.dL2FP1/nbdkit4.sock", "file.export": "/" } (qcow2)
    (100.00/100%)
[1179.3] Creating output metadata
[1181.1] Finishing off

1.2 Guest has disk on rhv4.3 after v2v conversion


2. Can't reproduce the problem when convert a windows guest to rhv4.4 via rhv and not install drivers for guest from virtio-win during v2v conversion

2.1#  virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1  esx7.0-win2019-x86_64 -b ovirtmgmt -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78 -o rhv -os 10.73.224.195:/home/nfs_export -of qcow2 -ip /home/passwd
[   0.0] Opening the source -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 esx7.0-win2019-x86_64 -it vddk  -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78
[   1.7] Creating an overlay to protect the source from being modified
[   2.7] Opening the overlay
[   9.8] Inspecting the overlay
[  15.2] Checking for sufficient free disk space in the guest
[  15.2] Estimating space required on target for each disk
[  15.2] Converting Windows Server 2019 Standard to run on KVM
virt-v2v: warning: /usr/share/virt-tools/pnp_wait.exe is missing.  
Firstboot scripts may conflict with PnP.
virt-v2v: warning: QEMU Guest Agent MSI not found on tools ISO/directory. 
You may want to install the guest agent manually after conversion.
virt-v2v: warning: there are no virtio drivers available for this version 
of Windows (10.0 x86_64 Server).  virt-v2v looks for drivers in 
/usr/share/virtio-win

The guest will be configured to use slower emulated devices.
virt-v2v: This guest does not have virtio drivers installed.
[  19.3] Mapping filesystem data to avoid copying unused and blank areas
[  20.4] Closing the overlay
[  20.7] Assigning disks to buses
[  20.7] Checking if the guest needs BIOS or UEFI to boot
[  20.7] Initializing the target -o rhv -os 10.73.224.195:/home/nfs_export
[  21.0] Copying disk 1/1 to /tmp/v2v.IEuB5E/e0c94e37-3753-44a5-9dfa-7f1dd63b9fbf/images/05c5747b-0d13-4bd8-81f3-911be441ca7d/89ed7a09-8505-442f-8ab9-e5fe1b40e784 (qcow2)
    (100.00/100%)
[ 510.4] Creating output metadata
[ 510.4] Finishing off

2.2 Import guest from export domain to data domain after v2v conversion, the guest has disk after importing

Comment 1 mxie@redhat.com 2021-01-07 07:35:35 UTC
Created attachment 1745203 [details]
vdsm.log

Comment 2 mxie@redhat.com 2021-01-07 07:37:19 UTC
Created attachment 1745210 [details]
engine.log

Comment 3 mxie@redhat.com 2021-01-07 07:38:50 UTC
Created attachment 1745211 [details]
virt-v2v-convert-windows-guest-rhv-upload-without-virtio-win.log

Comment 4 Tomáš Golembiovský 2021-01-08 19:55:02 UTC
This looks same as the bug we saw already in June.

- transfer is marked as finalized
- we fail to attach disk to VM because image is locked
- few seconds later transfer is again marked as finalized and lock is removed


Was there any bug opened on virt-v2v or on RHV regarding this at that time?

Comment 5 Xiaodai Wang 2021-01-09 04:35:36 UTC
(In reply to Tomáš Golembiovský from comment #4)
> This looks same as the bug we saw already in June.
> 
> - transfer is marked as finalized
> - we fail to attach disk to VM because image is locked
> - few seconds later transfer is again marked as finalized and lock is removed
> 
> 
> Was there any bug opened on virt-v2v or on RHV regarding this at that time?

There is a similar bug.
https://bugzilla.redhat.com/show_bug.cgi?id=1849861

Comment 6 Tomáš Golembiovský 2021-01-12 12:49:53 UTC
Thanks Xiaodai, marking as duplicate.

*** This bug has been marked as a duplicate of bug 1849861 ***

Comment 9 mxie@redhat.com 2021-01-13 01:31:01 UTC
I think they are different bugs, the failed reason of bug1913577 is "VAR__ACTION__ATTACH_ACTION_TO,VAR__TYPE__DISK,ACTION_TYPE_DISK_INTERFACE_UNSUPPORTED", but the failed reason of bug1849861 is "VAR__ACTION__ATTACH_ACTION_TO,VAR__TYPE__DISK,ACTION_TYPE_FAILED_DISK_IS_LOCKED", details info can be found in engine.logs of bugs. So bug1913577 should be reopen.


The engine.log of bug1913577:
.......
2021-01-06 14:20:46,472+08 INFO  [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (default task-39) [6cf842f3] Lock Acquired to object 'EngineLock:{exclusiveLocks='[8b968e5e-e3a2-47e4-a0e6-3305804fcd60=VM_DISK_BOOT, 19ed4885-90b3-42e0-9df8-3a022fccc3ce=DISK]', sharedLocks=''}'
2021-01-06 14:20:46,521+08 WARN  [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (default task-39) [6cf842f3] Validation of action 'AttachDiskToVm' failed for user admin@internal-authz. Reasons: VAR__ACTION__ATTACH_ACTION_TO,VAR__TYPE__DISK,ACTION_TYPE_DISK_INTERFACE_UNSUPPORTED,$osName Windows 2016 x64
2021-01-06 14:20:46,522+08 INFO  [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (default task-39) [6cf842f3] Lock freed to object 'EngineLock:{exclusiveLocks='[8b968e5e-e3a2-47e4-a0e6-3305804fcd60=VM_DISK_BOOT, 19ed4885-90b3-42e0-9df8-3a022fccc3ce=DISK]', sharedLocks=''}'
2021-01-06 14:20:46,527+08 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-39) [6cf842f3] EVENT_ID: VM_IMPORT_FROM_CONFIGURATION_ATTACH_DISKS_FAILED(175), VM esx7.0-win2019-x86_64 has been imported from the given configuration but the following disk(s) failed to attach: 19ed4885-90b3-42e0-9df8-3a022fccc3ce.
......



The engine.log of bug1849861:
......
2020-06-22 10:31:44,682+08 INFO  [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (default task-1287) [ea347a0] Failed to Acquire Lock to object 'EngineLock:{exclusiveLocks='[3695bc4e-a21f-4dde-ac86-2eb0966314ee=VM_DISK_BOOT, fd95ca53-bd87-423e-8ad3-c7a5e0b66a45=DISK]', sharedLocks=''}'
2020-06-22 10:31:44,683+08 WARN  [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (default task-1287) [ea347a0] Validation of action 'AttachDiskToVm' failed for user admin@internal-authz. Reasons: VAR__ACTION__ATTACH_ACTION_TO,VAR__TYPE__DISK,ACTION_TYPE_FAILED_DISK_IS_LOCKED
2020-06-22 10:31:44,692+08 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-1287) [ea347a0] EVENT_ID: VM_IMPORT_FROM_CONFIGURATION_ATTACH_DISKS_FAILED(175), VM ova_vm_nvV has been imported from the given configuration but the following disk(s) failed to attach: fd95ca53-bd87-423e-8ad3-c7a5e0b66a45.
.......

Comment 12 Klaus Heinrich Kiwi 2021-06-15 15:55:21 UTC
Richard,

 any updates or thoughts on how to progress on this one?

Comment 13 Richard W.M. Jones 2021-06-16 10:22:19 UTC
I missed this one, but the bottom line here is that our customers
all have virtio-win (so uninstalling it is not a valid test case),
and RHV only supports guests using virtio drivers, not the very slow
emulated drivers.  So even if we were to fix this to make it work
(assuming it is really a bug in v2v), it wouldn't be a supported
configuration in the end.

IOW this is not a bug.

Comment 14 mxie@redhat.com 2021-06-16 11:09:48 UTC
(In reply to Richard W.M. Jones from comment #13)
> I missed this one, but the bottom line here is that our customers
> all have virtio-win (so uninstalling it is not a valid test case),
> and RHV only supports guests using virtio drivers, not the very slow
> emulated drivers.  So even if we were to fix this to make it work
> (assuming it is really a bug in v2v), it wouldn't be a supported
> configuration in the end.
> 
> IOW this is not a bug.

If v2v converts guest without virtio-win is not a valid test case, I think v2v should depend on virtio-win in case customers forget to install virtio-win on v2v server

Comment 15 Richard W.M. Jones 2021-06-16 11:15:21 UTC
I'm slightly surprised that we don't have that dependency (even
on RHEL 8).  But yes I agree, we should have it, at least on RHEL.
Can you file a new bug about that - I'm doing a virt-v2v build
later today.


Note You need to log in before you can comment on or make changes to this bug.