Bug 2043333

Hi Rich,

I've determined that virtio-1.0 support level is impossible to *determine* for Windows guests.

I've spent today developing the following patch set:

     1  fa8bf7a05d72 convert/libosinfo: factor out v2v_osinfo_device_list_to_value_list()
     2  83e87ad7d3df convert/libosinfo: retrieve device list for OsinfoDeviceDriver
     3  fce3e8acc845 convert/libosinfo_utils: debug-log the devices supported by a driver
     4  07e2eb8f4f6a convert/libosinfo_utils: extract "best_driver" from "windows_virtio.ml"
     5  2f76a80b8100 convert/convert_windows: consult "best driver"'s dev list for virtio-1.0

(The purpose of this patch set would have been to reclassify the present BZ as a virt-v2v regression, from bug 1942325, and to solve the issue without touching osinfo-db.)

The first two patches extend the "osinfo_device_driver" type with the following field:

  devices : osinfo_device list

This provides the list of devices (in the usual, known format) that the driver supports.

Patch#3 is just for helping bug analysis -- it makes the debug log more verbose.

Patch#4 extracts the "best_driver" function from "copy_from_libosinfo" [convert/windows_virtio.ml]. Again, just refactoring. Here's the signature:

+val best_driver : Libosinfo.osinfo_device_driver list ->
+                  string ->
+                  Libosinfo.osinfo_device_driver
+(** [best_driver drivers arch] picks the best driver from [drivers] as follows:
+    - filters out drivers that:
+      - target a different architecture,
+      - are not pre-installable,
+      - have an invalid URL, or a non-local location;
+    - sorts the remaining drivers by priority, like libosinfo does;
+    - picks the top driver of the sorted list.
+    Raises Not_found if no driver in [drivers] survives filtering. *)

Patch#5 reworks "convert" in "convert/convert_windows.ml" as follows:
(a) don't just fetch the device list for the OS, but also fetch the driver list
(b) from the driver list, pick the "best driver", according to the above definition
(c) from the "best driver", grab the devices supported by that driver
(d) concatenate the device list from the "OS level" with the device list from the "best driver" (c), and pass this larger list to "os_support_of_osinfo_device_list", which determines Q35 and virtio-1.0 support.

Here's the shocking news (well, shocking to me). Even *before* my patch set, the *original* "copy_from_libosinfo" function does exactly nothing, because *no driver* from libosinfo survives the filtering. All drivers are removed by the criterion that says that the location (scheme) of the driver must be "file://". There is no such driver returned by libosinfo: all drivers are in the "https://" scheme (namely "https://fedorapeople.org/groups/virt/unattended/...").

Therefore, the actual virtio driver copying is performed by "copy_from_virtio_win".

Unfortunately, there is no information there that I could extract, and reuse, for determining virtio-1.0 support.

Corollary: it is impossible to determine virtio-1.0 support for Windows guests, given the current information present in the virtio-win ISO, *and* in osinfo-db. (I can't rely on any *driver-level* osinfo-db information for determining virtio-1.0 support if we then install *different* driver files from the virtio-win ISO.)

So, what I'm going to do is: submit a simple patch that simply assumes "true" for virtio-1.0 support, when a Windows guest is being converted. This field in gcaps is (intentionally) documented as follows:

  gcaps_virtio_1_0 : bool;
  (** The guest supports the virtio devices that it does at the virtio-1.0
      protocol level. *)

In other words, the field in itself does not say anything about virtio devices; it only has a meaning *IF* some other virtio-related gcaps field is "true" (such as gcaps_virtio_rng). THEN it means "yes, the OS can drive this virtio device even at the 1.0 protocol level". So, for Windows, we'll just assume that, IF we find drivers (on the virtio-win ISO) for a particular virtio device, THEN that driver is virtio-1.0 compatible.

diff --git a/convert/convert_windows.ml b/convert/convert_windows.ml
index 1c2d17f26543..5e2b8ad58b6a 100644
--- a/convert/convert_windows.ml
+++ b/convert/convert_windows.ml
@@ -248,7 +248,7 @@ let convert (g : G.guestfs) _ inspect _ static_ips =
              (Libosinfo_utils.string_of_osinfo_device_list devices);
            let { Libosinfo_utils.q35; vio10 } =
              Libosinfo_utils.os_support_of_osinfo_device_list devices in
-           (if q35 then Q35 else I440FX), vio10
+           (if q35 then Q35 else I440FX), (vio10 || true) (* RHBZ#2043333 *)
          with
          | Not_found ->
            (* Pivot on the year 2007.  Any Windows version from earlier than

... At this point, we could even remove "copy_from_libosinfo" altogether. :/

> Even *before* my patch set,
> the *original* "copy_from_libosinfo" function does exactly nothing, because *no
> driver* from libosinfo survives the filtering. All drivers are removed by the
> criterion that says that the location (scheme) of the driver must be "file://".

I'm in favour of just removing code that cannot do anything, and this
may be a case of that.  However the code was added by Pino for some
reason, so I wonder why.  The conversation upstream doesn't give much
hint (https://listman.redhat.com/archives/libguestfs/2020-January/thread.html)
Maybe it's possible to customize the osinfo db with local drivers?  Is
there an updated osinfo-db on the virtio-win disk?  ... Actually - yes
there is, see /usr/share/osinfo/os/microsoft.com/win-10.d/virtio-win-pre-installable-drivers-win-10.xml with file:/// URLs in it.  Maybe an extra step is needed
to connect this to the osinfo database?  The local database has some
info about PCI device IDs.

The approach (my patches) described at the top of comment 2 look viable after all, after Pino's explanation. There is namely, apparently, a packaging bug in Fedora's virtio-win: even though the drivers are available on the host filesystem as broken-out files, they are not advertised via /usr/share/osinfo/. The same bug does not seem to affect RHEL. My v2v dev env is Fedora 34, which is why the "best driver" logic in virt-v2v ends up with an empty set. I'll return to my patches. Thanks.

[Libguestfs] [v2v PATCH 0/7] Fix "virtio-transitional" regression for Windows guests
https://listman.redhat.com/archives/libguestfs/2022-January/msg00200.html
Message-Id: <20220128161218.14293-1-lersek>

(In reply to Laszlo Ersek from comment #5)
> [Libguestfs] [v2v PATCH 0/7] Fix "virtio-transitional" regression for Windows guests
> https://listman.redhat.com/archives/libguestfs/2022-January/msg00200.html
> Message-Id: <20220128161218.14293-1-lersek>

Sigh, I put the same bug in patch#7 that I fixed in patch#1. :/ From patch#7, the following hunks must be dropped:

 tests/test-v2v-cdrom.expected  |  2 +-
 tests/test-v2v-floppy.expected |  2 +-
 tests/test-v2v-i-ova.xml       |  8 ++++----

That's because the related test cases all contain

  export VIRTIO_WIN="$srcdir/../test-data/fake-virtio-win"

and so the "virtio-win-1.9.24-4.el8.noarch.rpm" package I have installed should be *ignored*.

I might have to sink the VIRTIO_WIN env var check in patch#1 into "copy_from_libosinfo", and then carry that check over to "best_driver" in patch #6.

For the record I ACKed most of the patches and (in a private email)
soft-ACKed the rest, but raised some concerns about whether we should
be using the libosinfo database for this.  However I'll let Laszlo
decide what we do.

[v2v PATCH v2 0/7] Fix "virtio-transitional" regression for Windows guests
Message-Id: <20220201123219.9317-1-lersek>
https://listman.redhat.com/archives/libguestfs/2022-February/msg00004.html

(In reply to Laszlo Ersek from comment #9)
> [v2v PATCH v2 0/7] Fix "virtio-transitional" regression for Windows guests
> Message-Id: <20220201123219.9317-1-lersek>
> https://listman.redhat.com/archives/libguestfs/2022-February/msg00004.html

Merged upstream as commit range 8abc07a8589a..cacedec64072.

Hi Ming Xie,

can you please grant qa_ack+? Thanks!
Laszlo

(In reply to Laszlo Ersek from comment #12)
> Hi Ming Xie,
> 
> can you please grant qa_ack+? Thanks!
> Laszlo

Done.

Verify the bug with below builds:
virt-v2v-1.45.97-4.el9.x86_64
libguestfs-1.46.1-2.el9.x86_64
guestfs-tools-1.46.1-6.el9.x86_64
libvirt-libs-8.0.0-3.el9.x86_64
qemu-img-6.2.0-7.el9.x86_64
nbdkit-1.28.5-1.el9.x86_64
libnbd-1.10.4-1.el9.x86_64


Steps:
1.Convert some windows guests from VMware to local libvirt by v2v 
# virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk7.0.2 -io  vddk-thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78  -ip /home/passwd $windows_guest

2.Check the libvirtxml of guests after conversion, there is no 'virtio-transitional' in xml
# virsh dumpxml esx7.0-win11-x86_64  |grep 'virtio-transitional'
noting
# virsh dumpxml esx7.0-win2012-x86_64  |grep 'virtio-transitional'
noting
# virsh dumpxml esx7.0-win2012r2-x86_64  |grep 'virtio-transitional'
noting
# virsh dumpxml esx7.0-win8.1-x86_64  |grep 'virtio-transitional'
noting
# virsh dumpxml esx7.0-win2022-x86_64  |grep 'virtio-transitional'
noting
# virsh dumpxml esx7.0-win8.1-i386 |grep 'virtio-transitional'
nothing

3.Check guests, checkpoints of esx7.0-win2022-x86_64, esx7.0-win11-x86_64, esx7.0-win8.1-i386 and esx7.0-win8.1-x86_64 guest are passed, esx7.0-win2012-x86_64 guest has bug1963032, esx7.0-win2012r2-x86_64 has bug1820152

Result:
  The bug has been fixed

Verify the bug with below builds
virt-v2v-1.45.98-1.el9.x86_64
libguestfs-1.46.1-2.el9.x86_64
guestfs-tools-1.46.1-6.el9.x86_64
libvirt-libs-8.0.0-4.el9.x86_64
qemu-img-6.2.0-7.el9.x86_64
virtio-win-1.9.19-5.el9_b.noarch

1.Convert a windows guests from VMware to json format by v2v 
# virt-v2v -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk7.0.2 -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA   esx6.7-win2022-x86_64-uefi -o json -os /home

2.Check if there is 'virtio-transitional' in json file after conversion
#cat /home/esx6.7-win2022-x86_64-uefi.json |grep 'virtio-transitional'
nothing

3.Convert a windows guests from VMware to local libvirt by v2v 
# virt-v2v -i vmx -it ssh ssh://root.196.89/vmfs/volumes/esx6.5-matrix/esx6.5-win11-x86_64/esx6.5-win11-x86_64.vmx -ip /home/esxpw 

4.Check if there is 'virtio-transitional' in guest libvirtxml after conversion
# virsh dumpxml esx6.5-win11-x86_64 |grep  'virtio-transitional'
nothing

Result:
  No problem found, move the bug from ON_QA to VERIFIED

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (new packages: virt-v2v), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2022:2566