RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1788823 - Virt-v2v firstboot scripts should run in order, with v2v network configuration happening first
Summary: Virt-v2v firstboot scripts should run in order, with v2v network configuratio...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: virt-v2v
Version: unspecified
Hardware: x86_64
OS: Unspecified
high
high
Target Milestone: beta
: ---
Assignee: Laszlo Ersek
QA Contact: mxie@redhat.com
URL:
Whiteboard: V2V
Depends On: 2059287
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-01-08 08:01 UTC by mxie@redhat.com
Modified: 2022-11-15 10:22 UTC (History)
8 users (show)

Fixed In Version: virt-v2v-2.0.7-1.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-11-15 09:55:44 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
firstboot-log.png (82.48 KB, image/png)
2020-01-08 08:01 UTC, mxie@redhat.com
no flags Details
fb.ml (1.22 KB, text/plain)
2022-03-07 13:32 UTC, Richard W.M. Jones
no flags Details
test artifacts from within the real, converted Windows 2019 Server guest (46.14 KB, application/x-xz)
2022-03-08 14:11 UTC, Laszlo Ersek
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2022:7968 0 None None None 2022-11-15 09:56:01 UTC

Description mxie@redhat.com 2020-01-08 08:01:43 UTC
Created attachment 1650599 [details]
firstboot-log.png

Description:
Make v2vnetcf run firstly in firstboot of windows guest after v2v conversion

Version-Release number of selected component (if applicable):
virt-v2v-1.40.2-16.module+el8.1.1+5309+6d656f05.x86_64
libguestfs-1.40.2-16.module+el8.1.1+5309+6d656f05.x86_64

How reducible:
100%

Reproducing steps:
1.Prepare a windows guest which has static IP, gateway, subnet mask and DNS
static ip: 192.168.122.11
subnet mask:255.255.248.0
default gateway:192.168.1.1
DNS server: 196.168.11.1

2. Check MAC address of the guest
# virsh -c vpx://root.73.141/data/10.73.75.219/?no_verify=1 dumpxml win2019-guest-static-ip-gw-subnet-dns
Enter root's password for 10.73.73.141: 
<domain type='vmware' xmlns:vmware='http://libvirt.org/schemas/domain/vmware/1.0'>
  <name>win2019-guest-static-ip-gw-subnet-dns</name>
....
  <interface type='bridge'>
      <mac address='00:50:56:ac:9a:7c'/>
      <source bridge='VM Network'/>
      <model type='e1000e'/>
    </interface>
....

3. Convert the guest from VMware to rhv, set static IP address,gateway, subnet mask and DNS in --mac option, which are consistent with the setting in guest
#  virt-v2v -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 -it vddk -io vddk-libdir=/home/vmware-vix-disklib-distrib -io  vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA  -o rhv-upload -oo rhv-cafile=/home/ca.pem -oo rhv-direct -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /home/rhvpasswd --password-file /home/passwd -of raw win2019-guest-static-ip-gw-subnet-dns -oo rhv-cluster=Default -os nfs_data --mac 00:50:56:ac:9a:7c:ip:192.168.122.11,192.168.1.1,21,196.168.11.1 -b ovirtmgmt -on win2019-static-ip
[   0.9] Opening the source -i libvirt -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 win2019-guest-static-ip-gw-subnet-dns -it vddk  -io vddk-libdir=/home/vmware-vix-disklib-distrib -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA
[   2.9] Creating an overlay to protect the source from being modified
[   3.4] Opening the overlay
[   8.3] Inspecting the overlay
[  17.2] Checking for sufficient free disk space in the guest
[  17.2] Estimating space required on target for each disk
[  17.2] Converting Windows Server 2019 Standard to run on KVM
virt-v2v: warning: /usr/share/virt-tools/pnp_wait.exe is missing.  
Firstboot scripts may conflict with PnP.
virt-v2v: warning: there is no QXL driver for this version of Windows (10.0 
x86_64).  virt-v2v looks for this driver in 
/usr/share/virtio-win/virtio-win.iso

The guest will be configured to use a basic VGA display driver.
virt-v2v: This guest has virtio drivers installed.
[  28.5] Mapping filesystem data to avoid copying unused and blank areas
[  29.5] Closing the overlay
[  29.6] Assigning disks to buses
[  29.6] Checking if the guest needs BIOS or UEFI to boot
[  29.6] Initializing the target -o rhv-upload -oc https://ibm-x3250m5-03.rhts.eng.pek2.redhat.com/ovirt-engine/api -op /home/rhvpasswd -os nfs_data
[  31.0] Copying disk 1/1 to qemu URI json:{ "file.driver": "nbd", "file.path": "/var/tmp/rhvupload.nKTzgR/nbdkit0.sock", "file.export": "/" } (raw)
    (100.00/100%)
[1566.5] Creating output metadata
[1568.2] Finishing off

4  Check the guest network after finishing conversion on rhv:
4.1 Power on and log into guest, found guest gets IP from DHCP at this time because firstboot.log shows rhev-apt is installing
4.2 But wait more than 30mins, found guest can get correct static IP, subnet mask, gateway and DNS IP


Actual result:
As above description


Expected result:
Because installing rhev-apt and uninstalling vmware-tools will take a long time sometimes, make v2vnetcf run firstly in firstboot of windows guest after v2v conversion in order to guest can get static ip quickly.


Additional info:

Comment 4 RHEL Program Management 2021-07-08 07:30:48 UTC
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Comment 5 Richard W.M. Jones 2021-07-08 07:51:39 UTC
My apologies again.  This bug was closed in error by an automated
process that we do not have control over.

Comment 6 Laszlo Ersek 2022-03-03 10:12:26 UTC
Hmmm, I'm not sure I agree about the bug report.

Powershell scripts (or more generally, firstboot scripts) are collected without any particular ordering. The "install_firstboot_powershell" function in "convert/windows.ml" calls "add_firstboot_script" in "common/mlcustomize/firstboot.ml", and the latter uses a simple counter "script_count" for "ordering" the scripts. The "order" is accidental (not inherent); it corresponds to the order of calls made to "add_firstboot_script".

If we get into dependency tracking, that's a much larger topic than just placing "v2vnetcf.ps1[.bat]" at the top.

> Because installing rhev-apt and uninstalling vmware-tools will take a long time sometimes, make v2vnetcf run firstly in firstboot of windows guest after v2v
> conversion in order to guest can get static ip quickly

I would say that the conversion cannot be considered complete until *all* firstboot scripts have run?

Opinions? Thanks.

Comment 7 Richard W.M. Jones 2022-03-04 09:01:54 UTC
As background for other readers, the context is our Firstboot module.
It has a single entrypoint to add a firstboot script (add_firstboot_script),
and scripts always run in the order that this is called:

https://github.com/libguestfs/libguestfs-common/blob/master/mlcustomize/firstboot.mli

As you describe, virt-v2v calls add_firstboot_script from several places, but
the resulting order is accidental.

So some practical problems arise (they may be slightly different from what
mxie wrote in her comment 0):

- Scripts cannot wait on previous actions to complete.  There is a case in
  virt-v2v where we want (see configure_network_interfaces):

  netkvm installed and operational
       |
       v
  run New-NetIPAddress commands

  (Netkvm is not installed by a script, it is installed as a side-effect
  of Windows starting up and finding new PCI devices with missing drivers,
  then locating the drivers in c:\windows\drivers\virtio.  This is something
  Windows does on its own somehow.)

- Scripts cannot wait on other scripts to complete, nor run in parallel:

  run New-NetIPAddress commands
       |                       \______
       v                              >
  uninstall stuff (VMware Tools etc)   install other tools (especially qemu-ga)

  (To avoid overcomplicating things, a simple numbering system would
  work fine here, eg. the early script can be numbered 100, and the
  later scripts can be numbered 200, and add_firstboot_script could just
  sort the scripts by number.  Also parallel running is a nice to have
  but not necessary.)

- Maybe as you said we want to delay user login until all firstboot scripts
  are complete, although I've no idea if this is possible.

- We cannot specify the shell that runs the script.  The default is
  batch scripts, and PowerShell scripts require a gross hack (see
  virt-v2v's install_firstboot_powershell).

Comment 8 Laszlo Ersek 2022-03-04 16:43:46 UTC
(In reply to Richard W.M. Jones from comment #7)

> - Scripts cannot wait on previous actions to complete.  There is a
>   case in virt-v2v where we want (see configure_network_interfaces):
>
>   netkvm installed and operational
>        |
>        v
>   run New-NetIPAddress commands
>
>   (Netkvm is not installed by a script, it is installed as a
>   side-effect of Windows starting up and finding new PCI devices with
>   missing drivers, then locating the drivers in
>   c:\windows\drivers\virtio.  This is something Windows does on its
>   own somehow.)

OK, I can understand this. In "configure_network_interfaces", we install
a firstboot (powershell) script that first polls for netkvm with
"Get-NetAdapter", and then runs "New-NetIPAddress".

IOW, I understand the arrow as a dependency above. OK.

> - Scripts cannot wait on other scripts to complete, nor run in
>   parallel:
>
>   run New-NetIPAddress commands
>        |                       \______
>        v                              >
>   uninstall stuff (VMware Tools etc)   install other tools (especially
>                                        qemu-ga)

I don't understand this diagram however...

Hm, OK!

First, I got briefly confused by "pnp_wait.exe", but then
<https://github.com/libguestfs/virt-v2v/commit/834bea6931b49dc4b14ebe04fd0e0884290bfb78>
explained the situation. There is no blocking there in practice.

Second, "configure_qemu_ga", "unconfigure_xenpv",
"unconfigure_prltools", "unconfigure_vmwaretools" are all based on
"add_firstboot_script". So this diagram above is what we "might want" as
the optimal dependency tree, but we cannot implement it for now.

So what we have in practice is this call tree, in
"convert/convert_windows.ml":

  do_convert
    configure_firstboot
      configure_wait_pnp
        add_firstboot_script
      configure_vmdp
        add_firstboot_script
      configure_qemu_ga
        add_firstboot_script
      unconfigure_xenpv
        add_firstboot_script
      unconfigure_prltools
        add_firstboot_script
      unconfigure_vmwaretools
        add_firstboot_script
    configure_network_interfaces
      install_firstboot_powershell "v2vnetcf.ps1"
        add_firstboot_script
          -> the script that polls for "netkvm.sys" with Get-NetAdapter,
             then calls New-NetIPAddress

which explains why the NIC configuration happens quite late.

(This call tree has been quite difficult to deduce -- the various shell
scripts in "convert/convert_linux.ml", and the various batch scripts in
"convert/convert_windows.ml", are embedded in the OCaml source without
any kind of indentation. This breaks the flow of the OCaml code and
makes it very difficult to read. I might re-indent all of these
script-inserts, with the help of trailing backslashes!)

>   (To avoid overcomplicating things, a simple numbering system would
>   work fine here, eg. the early script can be numbered 100, and the
>   later scripts can be numbered 200, and add_firstboot_script could
>   just sort the scripts by number.  Also parallel running is a nice to
>   have but not necessary.)

So this raises two questions (I'm ignoring parallelism at once):

- "add_firstboot_script" invokes "install_service" (either for Linux or
  Windows) on every call. Is that not waste? Can we use "lazy" or just a
  "ref" for installing the service only once?

  Writing the specific script to the script dir needs to be done on
  every call, of course.

- "firstboot_sh" and "firstboot_script" each use the (respective) "for"
  shell command, so that's where the scripts get ultimately ordered (by
  expansion of the "for" command). That's OK; what doesn't seem to fit
  with an early/late distinction is that "add_firstboot_script" could do
  any sorting *itself*. For sorting, "add_firstboot_script" would have
  to wait for the final script to be registered -- but we don't know
  when that happens.

  So I think we should have two counters (replacing "script_count") --
  an "early script counter" and a "late script counter". Then
  "add_firstboot_script" should take an extra boolean param (perhaps
  optional), stating if the script was "early". Then just increment the
  appropriate counter -- and assert that the early counter never reaches
  the starting value of the late counter (200). The actual sorting
  (execution of scripts in order) will happen in the guest, in
  "firstboot_sh" and "firstboot_script", just as before. There could be
  a gap between the early scripts and the late scripts, but that should
  be no problem.

I'm going to take a stab at this. Thanks for the help!

Comment 9 Richard W.M. Jones 2022-03-04 17:19:30 UTC
(In reply to Laszlo Ersek from comment #8)
> (In reply to Richard W.M. Jones from comment #7)
> >   run New-NetIPAddress commands
> >        |                       \______
> >        v                              >
> >   uninstall stuff (VMware Tools etc)   install other tools (especially
> >                                        qemu-ga)
> 
> I don't understand this diagram however...

Is this better?

          run New-NetIPAddress
         /                 \
        v                   v
   uninstall stuff        install stuff

Anyway it doesn't matter that they're parallel, just that when we
(un-)install things we already have a working network.  (It's more
important on Linux guests TBH, because there it'd be nice to have
working yum instead of using rpm -e / yum install localfile)

...
> Second, "configure_qemu_ga", "unconfigure_xenpv",
> "unconfigure_prltools", "unconfigure_vmwaretools" are all based on
> "add_firstboot_script". So this diagram above is what we "might want" as
> the optimal dependency tree, but we cannot implement it for now.

Right, it's the desired situation, not what we do now.

...
> (This call tree has been quite difficult to deduce -- the various shell
> scripts in "convert/convert_linux.ml", and the various batch scripts in
> "convert/convert_windows.ml", are embedded in the OCaml source without
> any kind of indentation. This breaks the flow of the OCaml code and
> makes it very difficult to read. I might re-indent all of these
> script-inserts, with the help of trailing backslashes!)

Sure, just go for it.

> >   (To avoid overcomplicating things, a simple numbering system would
> >   work fine here, eg. the early script can be numbered 100, and the
> >   later scripts can be numbered 200, and add_firstboot_script could
> >   just sort the scripts by number.  Also parallel running is a nice to
> >   have but not necessary.)
> 
> So this raises two questions (I'm ignoring parallelism at once):
> 
> - "add_firstboot_script" invokes "install_service" (either for Linux or
>   Windows) on every call. Is that not waste? Can we use "lazy" or just a
>   "ref" for installing the service only once?

Hmm, so it does.  Yes that's a bit of a waste, but I guess it's
only overwriting the same files.  But it'd be nice to fix that.

>   Writing the specific script to the script dir needs to be done on
>   every call, of course.
> 
> - "firstboot_sh" and "firstboot_script" each use the (respective) "for"
>   shell command, so that's where the scripts get ultimately ordered (by
>   expansion of the "for" command). That's OK; what doesn't seem to fit
>   with an early/late distinction is that "add_firstboot_script" could do
>   any sorting *itself*. For sorting, "add_firstboot_script" would have
>   to wait for the final script to be registered -- but we don't know
>   when that happens.

I haven't thought very deeply about it, but if add_firstboot_script
had an extra "priority" parameter, could it create the script as:

  let filename = sprintf "%04d-%04d-%s" priority !script_count (sanitize_name name)

and then they might naturally order themselves, with the script_count
just used for disambiguation?

>   So I think we should have two counters (replacing "script_count") --
>   an "early script counter" and a "late script counter". Then
>   "add_firstboot_script" should take an extra boolean param (perhaps
>   optional), stating if the script was "early". Then just increment the
>   appropriate counter -- and assert that the early counter never reaches
>   the starting value of the late counter (200). The actual sorting
>   (execution of scripts in order) will happen in the guest, in
>   "firstboot_sh" and "firstboot_script", just as before. There could be
>   a gap between the early scripts and the late scripts, but that should
>   be no problem.

OK, but would priority (a number) be better than just the flag?

Anyway, I'm not that fussed, whatever works.

> I'm going to take a stab at this. Thanks for the help!

Comment 10 Laszlo Ersek 2022-03-06 09:44:29 UTC
(In reply to Richard W.M. Jones from comment #9)

> I haven't thought very deeply about it, but if add_firstboot_script
> had an extra "priority" parameter, could it create the script as:
> 
>   let filename = sprintf "%04d-%04d-%s" priority !script_count
> (sanitize_name name)
> 
> and then they might naturally order themselves, with the script_count
> just used for disambiguation?

This is a great idea, thank you. The global counter will produce increasing integers (just not adjacent ones) within any given priority.

Comment 11 Laszlo Ersek 2022-03-07 11:55:19 UTC
(In reply to Richard W.M. Jones from comment #9)
> (In reply to Laszlo Ersek from comment #8)

> > (This call tree has been quite difficult to deduce -- the various shell
> > scripts in "convert/convert_linux.ml", and the various batch scripts in
> > "convert/convert_windows.ml", are embedded in the OCaml source without
> > any kind of indentation. This breaks the flow of the OCaml code and
> > makes it very difficult to read. I might re-indent all of these
> > script-inserts, with the help of trailing backslashes!)
> 
> Sure, just go for it.

Three hours and four patches later, I'm giving up. The worst offender is the powershell script embedded (as a string) in a DOS batch file embedded in an OCaml format string (commit d9dc6c42ae64, "convert/convert_windows.ml: Handle date formats with dots instead of /", 2021-11-12). Multiple levels of quoting, according to different rules of quoting (per level). Reformatting this correctly is basically impossible without regression testing, and I can't regression test (and can't ask QE to regression test) every single branch where I reformat a script or message. I guess we're stuck with these forever. :/

Comment 12 Richard W.M. Jones 2022-03-07 12:08:17 UTC
I was thinking over the weekend if there was some better way we could do this.

We currently have some Python scripts (see output/rhv-upload*.py) which get embedded
in the binary but are stored as separate files in the source.  The mechanism for
this is clunky to say the least, and I'm not actually a fan of storing these
scripts separately (I think it's easier to follow what they are doing if they
are inline with the code), but for these Python scripts, especially the nbdkit plugin
it makes a bit of sense.  Also allows syntax highlighting which is relevant for
Python but not really for batch files.

We effectively need some way to embed script, preserve tabs, allow insertion
of values safely (ie. sprintf + quote), yet not require too much complicated machinery.

Not sure I have any good answers.

BTW incremental improvements are good - maybe you didn't fix _every_ script, but
did you fix _some_ of the scripts?  That's a worthwhile incremental improvement.

Comment 13 Richard W.M. Jones 2022-03-07 13:27:25 UTC
Here are some other random ideas:

* Do a differential conversion before/after your patch using the phony
Windows guest (./run virt-v2v -i disk test-data/phony-guests/windows.img -o local -os .)
and check the contents of the files embedded in the output Windows image are
the same (using guestfish, virt-cat, virt-diff etc)

* One thing I mentioned above was:

> - We cannot specify the shell that runs the script.  The default is
>   batch scripts, and PowerShell scripts require a gross hack (see
>   virt-v2v's install_firstboot_powershell).

It'd be cool if add_firstboot_script had some kind of optional ?interpreter
parameter, and if interpreter was set to "Powershell" it did the
Powershell-into-batch encoding there.  That would simplify the double-encoding
problem which I agree is horrible.

* It's possible to test the firstboot functionality on its own without
virt-v2v, just to check that batch files and powershell scripts actually
run.  (You'll need a real Windows VM for this).  virt-customize could be used, 
but I think a custom program would be better for this.  I have a test script
which I'll attach separately.

Comment 14 Richard W.M. Jones 2022-03-07 13:32:06 UTC
Created attachment 1864346 [details]
fb.ml

Test firstboot script.  Compiling it is annoying!  Copy and paste
this very long command ...

$ ocamlfind ocamlopt -package guestfs,unix,str,gettext-stub -linkpkg -I common/mlcustomize -I common/mlpcre -I common/mltools -I common/mlstdutils -I common/mlutils -I common/mlgettext -I common/utils/.libs -I gnulib/lib/.libs mlgettext.cmxa mlstdutils.cmxa mlcutils.cmxa mlpcre.cmxa mltools.cmxa mlcustomize.cmxa fb.ml -cclib -lgnu -o fb

**NB** this will modify any Windows VM that you supply on the command line.

$ ./fb windows.img
Root device: /dev/sda2
Done

$ guestfish --ro -a windows.img -i
><fs> ll "/Program Files/Guestfs/Firstboot"
total 85
drwxrwxrwx 1 root root     0 Mar  7 13:28 .
drwxrwxrwx 1 root root     0 Mar  7 13:28 ..
-rwxrwxrwx 1 root root   638 Mar  7 13:28 firstboot.bat
-rwxrwxrwx 1 root root 82137 Mar  7 13:28 rhsrvany.exe
drwxrwxrwx 1 root root     0 Mar  7 13:28 scripts

><fs> ll "/Program Files/Guestfs/Firstboot/scripts"
total 1
drwxrwxrwx 1 root root  0 Mar  7 13:28 .
drwxrwxrwx 1 root root  0 Mar  7 13:28 ..
-rwxrwxrwx 1 root root 17 Mar  7 13:28 0001-hello.bat

><fs> cat "/Program Files/Guestfs/Firstboot/scripts/0001-hello.bat"
echo hello, world

Comment 15 Richard W.M. Jones 2022-03-07 13:37:14 UTC
(In reply to Richard W.M. Jones from comment #13)
> > - We cannot specify the shell that runs the script.  The default is
> >   batch scripts, and PowerShell scripts require a gross hack (see
> >   virt-v2v's install_firstboot_powershell).
> 
> It'd be cool if add_firstboot_script had some kind of optional ?interpreter
> parameter, and if interpreter was set to "Powershell" it did the
> Powershell-into-batch encoding there.  That would simplify the
> double-encoding
> problem which I agree is horrible.

To complete this thought ...  Could we copy the raw Powershell into
the guest with a ".ps1" extension?  I think firstboot.bat might
automatically run this as a Powershell script?  If not, firstboot.bat
could be modified to detect this and invoke Powershell.  This avoids
double encoding completely.

Comment 16 Laszlo Ersek 2022-03-07 14:19:35 UTC
(In reply to Richard W.M. Jones from comment #9)
> (In reply to Laszlo Ersek from comment #8)

> > - "add_firstboot_script" invokes "install_service" (either for Linux
> >   or Windows) on every call. Is that not waste? Can we use "lazy" or
> >   just a "ref" for installing the service only once?
>
> Hmm, so it does.  Yes that's a bit of a waste, but I guess it's
> only overwriting the same files.  But it'd be nice to fix that.

Looking at this in more depth, it's more complicated than I first
thought.

add_firstboot_script takes (before the addition of "prio") four
parameters, a usual invocation (from the MLI file) looks like
[add_firstboot_script g root name content], where "g" is the guestfs
instance (or connection), and "root" is the root dir from the
inspection.

Here's the problem: any caching within "add_firstboot_script" must be
specific to (g, root).

For explaining this stuff to myself, I've experimented with the
following example code in the toploop:

  open Printf;;

  let add1 a b c =
    let sum = lazy (a + b) in
    fprintf stdout "%b" (Lazy.is_val sum);
    Lazy.force sum + c;;

  let add2 a b =
    let sum = lazy (a + b) in
    fun c ->
      fprintf stdout "%b" (Lazy.is_val sum);
      Lazy.force sum + c;;

  let y1 = add1 3 4;;
  let y2 = add2 3 4;;

  y1 5;;
  y1 5;;
  y2 5;;
  y2 5;;

"y1" does not cache (a + b); "y2" caches (a + b). The difference is that
"add1 3 4" is a partial application that produces a function that
*contains* the lazy definition (so every invocation of that function
will create a new lazy), whereas "add2 3 4" produces a function *outside
of which* the lazy definition exists.

Furthermore, repeating the expression "add2 3 4 5" caches nothing
either, because the result of the (add 3 4) subexpression, which
contains the lazy value, is never preserved, but always evaluated from
scratch.

This means that I'd have to (1) split "add_firstboot_script" like "add2"
is above, but worse, (2) every *accidental* series of invocations, such
as

  add_firstboot_script g root name1 content1;
  ...
  add_firstboot_script g root name2 content2;

would have to be refactored as

  let add_fbs = add_firstboot_script g root in

  add_fbs name1 content1;
  ...
  add_fbs name2 content2;

which really is a total chore, as the whole point of the firstboot
scripts is that they don't need to know about each other :/

An alternative would be for the "install_service" functions (Linux,
Windows) to check the guest filesystem for the presence of the firstboot
service -- but that's perhaps more work than just putting the serive
(directories etc) in place!

So yes, now I can see why the code is the way it is. I think it's OK
after all.

Comment 17 Richard W.M. Jones 2022-03-07 14:53:29 UTC
You could add an opaque handle to the Firstboot module, eg:

firstboot.mli  -----

type t

val create : Guestfs.guestfs -> string -> t
val add_firstboot_script : t -> ...
val commit : t -> unit

firstboot.ml  -----

type t = Guestfs.guestfs * string
let create g root = g, root
let add_firstboot_script (g, root) ... =
let commit (g, root) =
  (* do all the one-time stuff *)

-----

Since it's called "commit" we might want it to not modify the filesystem
at all until you do the commit, although it wouldn't matter for virt-v2v
because virt-v2v will throw away the COW overlay on failure, and virt-customize
already half-writes changes on failure which is bad but this wouldn't be
any worse.  To have real commit-like semantics, save the firstboot scripts
into the (mutable) handle and write them only during commit.

Comment 18 Laszlo Ersek 2022-03-07 15:49:51 UTC
(In reply to Richard W.M. Jones from comment #12)

> We effectively need some way to embed script, preserve tabs, allow
> insertion of values safely (ie. sprintf + quote), yet not require too
> much complicated machinery.
>
> Not sure I have any good answers.

I had arrived at the same idea, and came up totally empty. I didn't even
find the "clunky" way to embed the python scripts...

> BTW incremental improvements are good - maybe you didn't fix _every_
> script, but did you fix _some_ of the scripts?  That's a worthwhile
> incremental improvement.

Yes, I did modify the Linux scripts, and some of the Windows ones. The
latter ones I threw away when I reached the powershell example, but I
guess I can redo that work.

(In reply to Richard W.M. Jones from comment #13)

> * Do a differential conversion before/after your patch using the phony
> Windows guest (./run virt-v2v -i disk
> test-data/phony-guests/windows.img -o local -os .) and check the
> contents of the files embedded in the output Windows image are the
> same (using guestfish, virt-cat, virt-diff etc)

This sounds good, thanks. A bit laborious but at least diffable...

>
> * One thing I mentioned above was:
>
> > - We cannot specify the shell that runs the script.  The default is
> >   batch scripts, and PowerShell scripts require a gross hack (see
> >   virt-v2v's install_firstboot_powershell).
>
> It'd be cool if add_firstboot_script had some kind of optional
> ?interpreter parameter, and if interpreter was set to "Powershell" it
> did the Powershell-into-batch encoding there.  That would simplify the
> double-encoding problem which I agree is horrible.

Currently we can install batch files (CMD scripts), and powershell
scripts that get launched from auto-generated batch files that use
"-ExecutionPolicy ByPass -file". However, the problem is that in
"configure_qemu_ga" we have a "mixed" script, one that runs some
commands as a normal BAT file for starters, then invokes 'powershell.exe
-command "..."'.:

>   echo Removing any previously scheduled qemu-ga installation
>   schtasks.exe /Delete /TN Firstboot-qemu-ga /F
>   echo Scheduling delayed installation of qemu-ga from %s
>   powershell.exe -command \"...\"

And that use case does not seem to be covered by the "specify
interpreter" idea -- it's a script that uses both interpreters
internally.

If we could rewrite the whole thing in PowerShell, then the existent
"install_firstboot_powershell" facility should suffice I think?

(

  BTW I could only find Filename.quote in OCaml, and that does not seem
  proper for quoting PowerShell scripts. For example, consider the OCaml
  snippet

> "powershell.exe -command \"... schtasks.exe ... /TR \\\"C:\\%s /forcerestart /qn /l+*vx C:\\%s.log\\\"\""

  if we strip the OCaml string literal escaping, we get the batch
  command

> powershell.exe -command "... schtasks.exe ... /TR \"C:\%s /forcerestart /qn /l+*vx C:\%s.log\""

  and this is where things fall apart. Consider the argument of the
  "-command" option. There, the quotes for the /TR option of
  schtasks.exe must be escaped, so we have \" instances, but at the same
  time, backslashes in the *pathnames* do not work as escape characters:

> -command "... /TR \"C:\%s /forcerestart /qn /l+*vx C:\%s.log\""
                    ^   ^                              ^      ^
                    |   |                              |      |
                    |   |                              |      escapes the subsequent quote
                    |   does not introduce an escape sequence, WTF
                    escapes the subsequent quote

  This is utterly inconsequential. I don't even know if quoting can be
  automated at all, like this.

)

... Ugh, wait a second! In relation to "configure_qemu_ga", thus far
I've only focused on commit d9dc6c42ae64 ("convert/convert_windows.ml:
Handle date formats with dots instead of /", 2021-11-12).

But what about the whole idea there, introduced in commit dc66e78fa37d
("windows: delay installation of qemu-ga MSI", 2020-03-10)?

commit dc66e78fa37db33e3c7358b7f7c7fa809cf62f9d
Author: Tomáš Golembiovský <tgolembi>
Date:   Thu Mar 5 15:37:13 2020 +0100

    windows: delay installation of qemu-ga MSI

    Instead of running firstboot script during early boot schedule a task
    delayed for 2 minutes.

    During the first boot, after virt-v2v conversion, Windows installs the
    drivers injected by virt-v2v. When this installation is finished
    Windows enforces some kind of internal reboot. This unfortunately
    terminates any running firstboot scripts thus killing the installation
    of qemu-ga MSI.

    This is just a best-effort mitigation. It can still happen (e.g. with
    slow disk drives) that the drivers are not yet installed when the
    delayed installation starts. On the other hand we cannot delay it too
    much otherwise we risk that the users logs in and will be doing some
    work when the MSI installation starts. After MSI installation finishes
    the VM needs to be rebooted which would be annoying if that would happen
    under users hands. Although this is not a best fix (that may come later
    as it is more complex, e.g. introducing waiting mechanism), the delay as
    it is defined works in most cases. And it dramaticaly improves the
    situations -- originaly I experienced more than 90% failure rate.

    Signed-off-by: Tomáš Golembiovský <tgolembi>

Something is off here. My understanding has been that "pnp_wait.exe" is
superfluous -- see
<https://github.com/libguestfs/virt-v2v/commit/834bea6931b49dc4b14ebe04fd0e0884290bfb78>.
However, the above-quoted commit appears to work around *precisely* the
lack of "pnp_wait.exe"!

So I'm getting the impression that we need a delicate order of firstboot
scripts:

- Let windows install the virtio drivers (via PnP); do not start other
  scripts until this completes. For this, restore "pnp_wait.exe"!

- After reboot, install the qemu-ga MSI. This triggers another automatic
  (?) reboot (but we're not exactly sure when?), so... block on this too
  somehow?

- No more reboots expected, so set up the static IPs (the subject of
  this BZ).

- Let the rest of the firstboot scripts run.

I'm highlighting this because the "powershell -command" mess could be
eliminated if we could resolve this *original* problem by assigning a
suitable priority to the qemu-ga MSI installation.

> * It's possible to test the firstboot functionality on its own without
> virt-v2v, just to check that batch files and powershell scripts
> actually run.  (You'll need a real Windows VM for this).
> virt-customize could be used, but I think a custom program would be
> better for this.  I have a test script which I'll attach separately.

Thanks, I've checked the script. I need to digest is for a bit. Because
I'd have to copy the CMD / powershell scripts over, and those are
parametric in virt-v2v (and conditional too), I'm worried that I'd make
mistakes in the "copying over". Diffing conversion results seems more
robust (although, in that case, I need to work on covering all
branches....)

(In reply to Richard W.M. Jones from comment #15)
> (In reply to Richard W.M. Jones from comment #13)
> > > - We cannot specify the shell that runs the script.  The default
> > >   is batch scripts, and PowerShell scripts require a gross hack
> > >   (see virt-v2v's install_firstboot_powershell).
> >
> > It'd be cool if add_firstboot_script had some kind of optional
> > ?interpreter parameter, and if interpreter was set to "Powershell"
> > it did the Powershell-into-batch encoding there.  That would
> > simplify the double-encoding problem which I agree is horrible.
>
> To complete this thought ...  Could we copy the raw Powershell into
> the guest with a ".ps1" extension?  I think firstboot.bat might
> automatically run this as a Powershell script?  If not, firstboot.bat
> could be modified to detect this and invoke Powershell.

The following pattern does not work correctly:

  @echo off
  for %%F in (*.bat *.ps1) do (echo %%F)

Because it groups files based on the pattern:

  01.bat
  03.bat
  02.ps1
  04.ps1

But the following works:

  @echo off
  for %%F in (??.*) do (
    if %%~xF == .bat (
      echo run %%F as a batch file
    ) else (
      if  %%~xF == .ps1 (
        echo run %%F as a PS script
      )
    )

Producing the output

  run 01.bat as a batch file
  run 02.ps1 as a PS script
  run 03.bat as a batch file
  run 04.ps1 as a PS script

Where we could reuse "powershell.exe -ExecutionPolicy ByPass -file ..."
for the ps1 files, from "install_firstboot_powershell".

The question is whether the powershell scripts can live in the same
directory as the batch files, or if there's a special reason for keeping
them in "windows-system-root\temp".

> This avoids double encoding completely.

The double quoting is not related to this; that's related to running
powershell -command "..." from a batch file.

In summary:

- I'll resume work on indenting the scripts.

- I'll try to verify the results by comparing the pre-patch & post-patch
  scripts that are generated during the conversion of the phony windows
  guest.

- I'll attempt to rewrite "configure_qemu_ga" in pure powershell at
  first. That should take care of the horrible nested quoting (thus far
  I can see only one instance of this in virt-v2v).

- Think more about replacing the whole "schedule qemu-ga MSI install 2
  minutes into the future" with a fitting priority.

Comment 19 Richard W.M. Jones 2022-03-07 16:04:28 UTC
Re quoting: The OCaml Filename.quote function only works for Unix shell scripts.
If Windows is any different then we'd need to write our own quote function.

Re pnp_wait: We don't package this for RHEL, but it is open source and part of
the rhsrvany project which we do build for RHEL (for rhsrvany.exe) so there
is no barrier to us using pnp_wait if we need it (I wasn't aware before of what
it did before now).  It's also possible that
https://github.com/libguestfs/virt-v2v/commit/834bea6931b49dc4b14ebe04fd0e08842
is simply wrong!

> The question is whether the powershell scripts can live in the same
> directory as the batch files, or if there's a special reason for keeping
> them in "windows-system-root\temp".

Putting them in the same directory sounds right.  They're only in %systemroot%\Temp
now for the bogus reason that the virt-v2v code doesn't know about the
\Program Files\Guestfs\Firstboot path which is buried inside the Firstboot
module.  (If Firstboot handled both batch and powershell, it could put both
types of scripts in the same place).

Comment 20 Laszlo Ersek 2022-03-08 09:52:55 UTC
(In reply to Richard W.M. Jones from comment #19)

> > The question is whether the powershell scripts can live in the same
> > directory as the batch files, or if there's a special reason for
> > keeping them in "windows-system-root\temp".
> 
> Putting them in the same directory sounds right.  They're only in
> %systemroot%\Temp now for the bogus reason that the virt-v2v code
> doesn't know about the \Program Files\Guestfs\Firstboot path which is
> buried inside the Firstboot module.  (If Firstboot handled both batch
> and powershell, it could put both types of scripts in the same place).

I'd like to postpone this for now.

Adding an "interpreter" parameter to "Firstboot.add_firstboot_script"
would create a leaky abstraction:

- For Linux guests, the interpreter of each firstboot script is given
inside the script, with the shebang. Therefore the "interpreter"
(optional) parameter would not make any sense, plus it would not (should
not) control the suffix ("extension") when the script is copied into the
guest.

- Fow Windows guests, the interpreter would be passed as a parameter
(Cmd (default) versus PowerShell), and it would control the extension we
appended to the filename (.bat vs. .ps1).

This conflicts with "Firstboot.add_firstboot_script" striving to handle
the Linux vs. Windows distinction internally.

The requirement to provide the interpreter externally is specific to
Windows, and so for now I'd like to preserve
"Windows.install_firstboot_powershell" in virt-v2v.

Comment 21 Laszlo Ersek 2022-03-08 11:50:48 UTC
What package do I need to install (on Fedora) to get "/usr/share/virt-tools/rhsrvany.exe"? Thanks!

Comment 22 Richard W.M. Jones 2022-03-08 12:13:07 UTC
So note that /usr/share/virt-tools/rhsrvany.exe is a symlink.

To get that symlink, install virt-v2v:

# dnf install /usr/share/virt-tools/rhsrvany.exe
Last metadata expiration check: 0:28:15 ago on Tue 08 Mar 2022 11:39:22 GMT.
Package virt-v2v-1:1.45.99-1.fc37.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!

But assuming you actually mean the file it links to
(/usr/i686-w64-mingw32/sys-root/mingw/bin/rhsrvany.exe) then in both
recent Fedora and RHEL 9 that file is provided by mingw32-srvany
(source: mingw-srvany) which is cross-compiled from source using the
Fedora Windows cross compiler toolchain.  (In RHEL <= 8 we bundled
rhsrvany.exe as a binary).

Comment 23 Laszlo Ersek 2022-03-08 12:29:45 UTC
Thank you; after installing mingw32-srvany, VIRT_TOOLS_DATA_DIR=/usr/i686-w64-mingw32/sys-root/mingw/bin does the trick!

Comment 24 Laszlo Ersek 2022-03-08 13:10:26 UTC
Sanity checks done with differential "phony" conversion, and comparing
files (using guestfish) between the output disks:

$ ( cd /tmp/before && find . -type f | sort )

./Firstboot/firstboot.bat
./Firstboot/pnp_wait.exe
./Firstboot/rhsrvany.exe
./Firstboot/scripts/5000-0001-wait-pnp.bat
./Firstboot/scripts/5000-0002-install-qemu-ga-i386-msi.bat
./Firstboot/scripts/5000-0003-v2vnetcf-ps1.bat
./Temp/v2vnetcf.ps1

$ ( cd /tmp/after && find . -type f | sort )

./Firstboot/firstboot.bat
./Firstboot/pnp_wait.exe
./Firstboot/rhsrvany.exe
./Firstboot/scripts/2500-0003-v2vnetcf-ps1.bat
./Firstboot/scripts/5000-0001-wait-pnp.bat
./Firstboot/scripts/5000-0002-install-qemu-ga-i386-msi-ps1.bat
./Temp/install-qemu-ga-i386.msi.ps1
./Temp/v2vnetcf.ps1

(Note that the priority group prefix is present on the "before" state as
well, given that the patch for the Firstboot module implementing it is
part of my libguestfs-common tree in both cases.)

- The files with identical relative pathnames have identical contents.

- "5000-0003-v2vnetcf-ps1.bat" has been renamed to
  "2500-0003-v2vnetcf-ps1.bat", implementing the change requested in
  this BZ. The contents of these files are identical, namely:

> /Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy ByPass -file /Windows\Temp\v2vnetcf.ps1

- Due to the qga install script rewrite, the file
  "5000-0002-install-qemu-ga-i386-msi.bat" has been replaced with
  "5000-0002-install-qemu-ga-i386-msi-ps1.bat":

> --- before/Firstboot/scripts/5000-0002-install-qemu-ga-i386-msi.bat     2022-03-08 13:45:58.000000000 +0100
> +++ after/Firstboot/scripts/5000-0002-install-qemu-ga-i386-msi-ps1.bat  2022-03-08 13:46:56.000000000 +0100
> @@ -1,5 +1 @@
> -echo Removing any previously scheduled qemu-ga installation
> -schtasks.exe /Delete /TN Firstboot-qemu-ga /F
> -echo Scheduling delayed installation of qemu-ga from qemu-ga-i386.msi
> -powershell.exe -command "$d = (get-date).AddSeconds(120); $FormatHack = (($([System.Globalization.DateTimeFormatInfo]::CurrentInfo.ShortDatePattern) -replace 'y+', 'yyyy') -replace 'M+', 'MM') -replace 'd+', 'dd'; schtasks.exe /Create /SC ONCE /ST $d.ToString('HH:mm') /SD $d.ToString($FormatHack) /RU SYSTEM /TN Firstboot-qemu-ga /TR \"C:\qemu-ga-i386.msi /forcerestart /qn /l+*vx C:\qemu-ga-i386.msi.log\""
> -
> \ No newline at end of file
> +/Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy ByPass -file /Windows\Temp\install-qemu-ga-i386.msi.ps1
> \ No newline at end of file

- And the new PS1 script "install-qemu-ga-i386.msi.ps1" contains:

> # Uncomment this line for lots of debug output.
> # Set-PSDebug -Trace 2
>
> echo Removing any previously scheduled qemu-ga installation
> schtasks.exe /Delete /TN Firstboot-qemu-ga /F
>
> echo Scheduling delayed installation of qemu-ga from qemu-ga-i386.msi
> $d = (get-date).AddSeconds(120)
> $dtfinfo = [System.Globalization.DateTimeFormatInfo]::CurrentInfo
> $sdp = $dtfinfo.ShortDatePattern
> $sdp = $sdp -replace 'y+', 'yyyy'
> $sdp = $sdp -replace 'M+', 'MM'
> $sdp = $sdp -replace 'd+', 'dd'
> schtasks.exe /Create /SC ONCE `
>   /ST $d.ToString('HH:mm') /SD $d.ToString($sdp) `
>   /RU SYSTEM /TN Firstboot-qemu-ga `
>   /TR "C:\qemu-ga-i386.msi /forcerestart /qn /l+*vx C:\qemu-ga-i386.msi.log"

I've also installed a real Win2019 Server domain, so I'm going to
regression-test the qga install rework, and the static IP config
reordering, on that guest.

Comment 25 Laszlo Ersek 2022-03-08 14:11:41 UTC
Created attachment 1864620 [details]
test artifacts from within the real, converted Windows 2019 Server guest

(replaced "echo" with "Write-Host" in the qga install script)

Good test results with the real Windows 2019 Server guest; the static IP config took effect immediately (checked right after login with "ipconfig /all"), and the qga MSI was also installed correctly (after 2 minutes, followed by automatic reboot); see the attachment.

Comment 26 Laszlo Ersek 2022-03-08 14:31:44 UTC
[libguestfs-common PATCH] mlcustomize/firstboot: take an optional priority in "add_firstboot_script"
Message-Id: <20220308142922.6267-1-lersek>
https://listman.redhat.com/archives/libguestfs/2022-March/028377.html

[v2v PATCH 00/12] Windows: set static IP addresses as first thing at first boot
Message-Id: <20220308143059.6384-1-lersek>
https://listman.redhat.com/archives/libguestfs/2022-March/028378.html

Comment 27 Richard W.M. Jones 2022-03-09 09:04:29 UTC
(In reply to Laszlo Ersek from comment #25)
> Created attachment 1864620 [details]
> test artifacts from within the real, converted Windows 2019 Server guest
> 
> (replaced "echo" with "Write-Host" in the qga install script)
> 
> Good test results with the real Windows 2019 Server guest; the static IP
> config took effect immediately (checked right after login with "ipconfig
> /all"), and the qga MSI was also installed correctly (after 2 minutes,
> followed by automatic reboot); see the attachment.

I see from the artifacts that pnp_wait.exe was used.

I checked now and we do in fact have pnp_wait.exe already in RHEL 9.0+
(provided by mingw32-srvany).

However in virt-v2v we do not create any symlink from /usr/share/virt-tools
to /usr/i686-w64-mingw32/sys-root/mingw/bin/pnp_wait.exe (different
from what we do with rhsrvany).  I guess we'll need to adjust packaging
to make this symlink?

Comment 28 Laszlo Ersek 2022-03-09 09:57:28 UTC
I'm still undecided if "pnp_wait.exe" makes any difference in practice. In my test environment (on Fedora), "pnp_wait.exe" was picked up together with "rhsrvany.exe" because I had to set "VIRT_TOOLS_DATA_DIR=/usr/i686-w64-mingw32/sys-root/mingw/bin" for the latter (comment 23) -- and it exposed the former too (unexpectedly to me -- I had not reviewed that part of the source code, for locating "pnp_wait.exe" --, but without any ill effects).

Now, in the log (comment 25), "artifacts/Program Files/Guestfs/Firstboot/scripts-done/5000-0001-wait-pnp.log" contains

> start waiting for PnP to complete @ Tue Mar 08 06:04:47 2022
> done waiting for PnP to complete @ Tue Mar 08 06:04:47 2022

that's basically a sub-second interval, and I'm not sure if there was really anything to wait for!

Either way, I think it would be safer for us to resume using "pnp_wait.exe", and so, indeed a new symlink in the spec file would be good.

Comment 29 Richard W.M. Jones 2022-03-09 11:52:46 UTC
Added to Fedora:
https://src.fedoraproject.org/rpms/virt-v2v/c/1dc352f4e465c85fad41989cfab633f05fc62fac?branch=rawhide

When I synch the spec file with Rawhide in RHEL 9.1 (bug 2059287)
this means we should get this change.

Comment 30 Laszlo Ersek 2022-03-09 14:47:11 UTC
(In reply to Laszlo Ersek from comment #26)
> [libguestfs-common PATCH] mlcustomize/firstboot: take an optional priority in "add_firstboot_script"
> Message-Id: <20220308142922.6267-1-lersek>
> https://listman.redhat.com/archives/libguestfs/2022-March/028377.html

libguestfs-common commit ab708d11d832.

> [v2v PATCH 00/12] Windows: set static IP addresses as first thing at first boot
> Message-Id: <20220308143059.6384-1-lersek>
> https://listman.redhat.com/archives/libguestfs/2022-March/028378.html

virt-v2v commit range 3a66d0fece08..5d24c1575750.

Comment 33 mxie@redhat.com 2022-03-23 02:00:05 UTC
Test the bug with below builds:
virt-v2v-2.0.0-2.el9.x86_64
libguestfs-1.48.0-2.el9.x86_64
libvirt-libs-8.1.0-1.el9.x86_64
qemu-img-6.2.0-11.el9_0.1.x86_64
nbdkit-server-1.30.1-1.el9.x86_64
libnbd-1.12.2-1.el9.x86_64


Steps:

Scenario1:
1.1 Prepare a windows which has been installed vmware-tools and its network gets IP from DHCP and on VMware
# virsh -c vpx://root.198.169/data/10.73.199.217/?no_verify=1  dumpxml esx7.0-win2019-network-dhcp
....
 <interface type='bridge'>
      <mac address='00:50:56:83:90:83' type='generated'/>
      <source bridge='VM Network'/>
      <model type='e1000e'/>
    </interface>
.....

1.2 Convert the guest from VMware and set static ip, gateway, subnet mask and dns with --mac option by v2v
# virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk7.0.3 -io  vddk-thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78  -o rhv-upload -of qcow2 -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api  -op /home/rhvpasswd  -os nfs_data -b ovirtmgmt -ip /home/passwd esx7.0-win2019-network-dhcp --mac 00:50:56:83:90:83:ip:192.168.24.3,192.168.24.2,22,192.168.24.1
[   0.0] Setting up the source: -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 -it vddk esx7.0-win2019-network-dhcp
[   2.0] Opening the source
[   7.8] Inspecting the source
[  11.8] Checking for sufficient free disk space in the guest
[  11.8] Converting Windows Server 2019 Standard to run on KVM
virt-v2v: This guest has virtio drivers installed.
[  21.6] Mapping filesystem data to avoid copying unused and blank areas
[  22.7] Closing the overlay
[  23.0] Assigning disks to buses
[  23.0] Checking if the guest needs BIOS or UEFI to boot
[  23.0] Setting up the destination: -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -os nfs_data
[  35.6] Copying disk 1/1
█ 100% [****************************************]
[ 697.3] Creating output metadata
[ 707.0] Finishing off


1.3 Check the guest after finising conversion, get below result:
1.3.1 Related virtio drivers have been installed successfully
1.3.2 Check the firstboot log:
1) "C:\Program Files\Guestfs\Firstboot\scripts\2500-0004-v2vnetcf-ps1.bat" will run firstly, and static ip, gateway, subnet mask and dns are set for network port correctly
2) "C:\Program Files\Guestfs\Firstboot\scripts\5000-0001-wait-pnp.bat" wil run secondly, exit code is 0
3) "C:\Program Files\Guestfs\Firstboot\scripts\5000-0002-install-qemu-ga-x86_64-msi-ps1.bat" will run thirdly, exit code is 0, and check qemu-ga service is running normally
4) C:\Program Files\Guestfs\Firstboot\scripts\5000-0003-uninstall-VMware-Tools.bat" will run fourthly, but exit code is 1603, the problem has been tracked by bug1988279
5) firstboot service is uninstalled successfully


Scenario2:
2.1 Prepare a windows which has static IP and vmware-tools installed on VMware
# virsh -c vpx://root.73.141/data/10.73.75.219/?no_verify=1 dumpxml esx6.7-win2019-static-ip
.....
    <interface type='bridge'>
      <mac address='00:50:56:83:2e:62' type='static'/>
      <source bridge='VM Network'/>
      <model type='e1000e'/>
    </interface>
......

2.2 Convert the guest from VMware and set static ip, gateway, subnet mask and dns with --mac option by v2v
#  virt-v2v -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk7.0.3 -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api  -op /home/rhvpasswd  -os nfs_data -b ovirtmgmt -ip /home/passwd  esx6.7-win2019-static-ip --mac 00:50:56:83:2e:62:ip:192.168.100.20,192.168.100.10,24,192.168.100.1
[   0.1] Setting up the source: -i libvirt -ic vpx://root.73.141/data/10.73.75.219/?no_verify=1 -it vddk esx6.7-win2019-static-ip
[   2.1] Opening the source
[   7.0] Inspecting the source
[  12.0] Checking for sufficient free disk space in the guest
[  12.0] Converting Windows Server 2019 Standard to run on KVM
virt-v2v: This guest has virtio drivers installed.
[  21.4] Mapping filesystem data to avoid copying unused and blank areas
[  22.7] Closing the overlay
[  23.0] Assigning disks to buses
[  23.0] Checking if the guest needs BIOS or UEFI to boot
[  23.0] Setting up the destination: -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -os nfs_data
[  42.9] Copying disk 1/1
█ 100% [****************************************]
[ 699.6] Creating output metadata
[ 705.2] Finishing off

2.3 Check the guest after finising conversion, get below result:
2.3.1 Related virtio drivers have been installed successfully
2.3.2 Check the firstboot log:
1) "C:\Program Files\Guestfs\Firstboot\scripts\2500-0004-v2vnetcf-ps1.bat" will run firstly, and static ip, gateway, subnet mask and dns are set for network port correctly
2) "C:\Program Files\Guestfs\Firstboot\scripts\5000-0001-wait-pnp.bat" wil run secondly, exit code is 0
3) "C:\Program Files\Guestfs\Firstboot\scripts\5000-0002-install-qemu-ga-x86_64-msi-ps1.bat" will run thirdly, exit code is 0, but qemu-ga service is not running because of system error 258, the problem has been tracked by bug1963032
4) C:\Program Files\Guestfs\Firstboot\scripts\5000-0003-uninstall-VMware-Tools.bat" will run fourthly, exit code is 0
5) firstboot service is uninstalled successfully

Result:
   The bug has been fixed

Comment 36 mxie@redhat.com 2022-03-25 04:47:53 UTC
Verify the bug with below builds:
virt-v2v-2.0.1-1.el9.x86_64
libguestfs-1.48.0-2.el9.x86_64
guestfs-tools-1.48.0-1.el9.x86_64
libvirt-libs-8.1.0-1.el9.x86_64
qemu-img-6.2.0-12.el9.x86_64
nbdkit-server-1.30.1-2.el9.x86_64
libnbd-1.12.2-1.el9.x86_64

Steps:
Scenario1:
1.1 Prepare a windows which has been installed vmware-tools and its network gets IP from DHCP and on VMware
# virsh -c vpx://root.198.169/data/10.73.199.217/?no_verify=1  dumpxml esx7.0-win11-vmware-tools-dhcp-network
....
    <interface type='bridge'>
      <mac address='00:50:56:83:6b:2e' type='generated'/>
      <source bridge='VM Network'/>
      <model type='e1000e'/>
    </interface>
    <interface type='bridge'>
      <mac address='00:50:56:83:b0:71' type='generated'/>
      <source bridge='VM Network'/>
      <model type='vmxnet3'/>
    </interface>
    <interface type='bridge'>
      <mac address='00:50:56:83:f1:4d' type='generated'/>
      <source bridge='VM Network'/>
      <model type='e1000'/>
    </interface>
.....

1.2 Convert the guest from VMware and set static ip, gateway, subnet mask and dns with --mac option by v2v
# virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk7.0.3 -io  vddk-thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78  -o rhv-upload -of qcow2 -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api  -op /home/rhvpasswd  -os nfs_data -b ovirtmgmt -ip /home/passwd esx7.0-win11-vmware-tools-dhcp-network --mac 00:50:56:83:6b:2e:ip:192.168.24.3,192.168.24.2,22,192.168.24.1  --mac 00:50:56:83:b0:71:ip:192.168.100.3,192.168.100.2,24,192.168.100.1 --mac 00:50:56:83:f1:4d:ip:192.168.80.3,192.168.80.2,23,192.168.80.1
[   0.8] Setting up the source: -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 -it vddk esx7.0-win11-vmware-tools-dhcp-network
[   2.7] Opening the source
[   7.8] Inspecting the source
[  12.3] Checking for sufficient free disk space in the guest
[  12.3] Converting Windows 10 Enterprise to run on KVM
virt-v2v: This guest has virtio drivers installed.
[  17.3] Mapping filesystem data to avoid copying unused and blank areas
[  18.9] Closing the overlay
[  19.1] Assigning disks to buses
[  19.1] Checking if the guest needs BIOS or UEFI to boot
[  19.1] Setting up the destination: -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -os nfs_data
[  40.8] Copying disk 1/1
█ 100% [****************************************]
[ 901.7] Creating output metadata
[ 912.4] Finishing off

1.3 Check the guest after finishing conversion, get below result:
1.3.1 Related virtio drivers have been installed successfully
1.3.2 Check the firstboot log:
1) "C:\Program Files\Guestfs\Firstboot\scripts\2500-0004-v2vnetcf-ps1.bat" will run firstly, but only one network can get correct static ip, gateway, subnet mask and dns, the problem has been tracked by bug2068361
2) "C:\Program Files\Guestfs\Firstboot\scripts\5000-0001-wait-pnp.bat" wil run secondly, exit code is 0
3) "C:\Program Files\Guestfs\Firstboot\scripts\5000-0002-install-qemu-ga-x86_64-msi-ps1.bat" will run thirdly, exit code is 0, and check qemu-ga service is running normally
4) C:\Program Files\Guestfs\Firstboot\scripts\5000-0003-uninstall-VMware-Tools.bat" will run fourthly, but exit code is 1603, the problem has been tracked by bug1988279
5) firstboot service is uninstalled successfully


Scenario2:
2.1 Prepare a windows which has static IP and vmware-tools installed on VMware
# virsh -c vpx://root.73.141/data/10.73.196.89/?no_verify=1 dumpxml esx6.5-win2022-x86_64-vmware-tools-static-ip
.....
  <interface type='bridge'>
      <mac address='00:50:56:ac:55:cb' type='generated'/>
      <source bridge='VM Network'/>
      <model type='e1000e'/>
    </interface>
......

2.2 Convert the guest from VMware and set static ip, gateway, subnet mask and dns with --mac option by v2v
# virt-v2v -ic vpx://root.73.141/data/10.73.196.89/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk7.0.3 -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api  -op /home/rhvpasswd  -os nfs_data -b ovirtmgmt -ip /home/passwd  esx6.5-win2022-x86_64-vmware-tools-static-ip --mac 00:50:56:ac:55:cb:ip:10.73.100.100,10.73.100.10,24,10.73.100.1
[   0.0] Setting up the source: -i libvirt -ic vpx://root.73.141/data/10.73.196.89/?no_verify=1 -it vddk esx6.5-win2022-x86_64-vmware-tools-static-ip
[   1.9] Opening the source
[  10.7] Inspecting the source
[  17.1] Checking for sufficient free disk space in the guest
[  17.1] Converting Windows Server 2022 Standard to run on KVM
virt-v2v: This guest has virtio drivers installed.
[  26.9] Mapping filesystem data to avoid copying unused and blank areas
[  28.5] Closing the overlay
[  28.7] Assigning disks to buses
[  28.7] Checking if the guest needs BIOS or UEFI to boot
[  28.7] Setting up the destination: -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -os nfs_data
[  53.3] Copying disk 1/1
█ 100% [****************************************]
[ 765.3] Creating output metadata
[ 774.0] Finishing off

2.3 Check the guest after finishing conversion, get below result:
2.3.1 Related virtio drivers have been installed successfully
2.3.2 Check the firstboot log:
1) "C:\Program Files\Guestfs\Firstboot\scripts\2500-0004-v2vnetcf-ps1.bat" will run firstly, and static ip, gateway, subnet mask and dns are set for network port correctly
2) "C:\Program Files\Guestfs\Firstboot\scripts\5000-0001-wait-pnp.bat" wil run secondly, exit code is 0
3) "C:\Program Files\Guestfs\Firstboot\scripts\5000-0002-install-qemu-ga-x86_64-msi-ps1.bat" will run thirdly, exit code is 0, but qemu-ga service is not running because of system error 258, the problem has been tracked by bug1963032
4) C:\Program Files\Guestfs\Firstboot\scripts\5000-0003-uninstall-VMware-Tools.bat" will run fourthly, exit code is 0
5) firstboot service is uninstalled successfully


Result:
   The bug has been fixed, move the bug from ON_QA to VERIFIED

Comment 38 errata-xmlrpc 2022-11-15 09:55:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: virt-v2v security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7968


Note You need to log in before you can comment on or make changes to this bug.