Bug 1306586 - change Windows drivers and sysprep delivery method to a CDROM
Summary: change Windows drivers and sysprep delivery method to a CDROM
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Virt
Version: 3.6.2
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ovirt-4.4.0
: ---
Assignee: Steven Rosenberg
QA Contact: Beni Pelled
URL:
Whiteboard:
Depends On:
Blocks: 1232559 1305900 1533540
TreeView+ depends on / blocked
 
Reported: 2016-02-11 11:34 UTC by Michal Skrivanek
Modified: 2020-08-05 06:10 UTC (History)
23 users (show)

Fixed In Version: rhv-4.4.0-35
Doc Type: Enhancement
Doc Text:
The floppy device has been replaced by a CDROM device for sysprep installation of Compatibility Versions 4.4 and later.
Clone Of: 1300959
Environment:
Last Closed: 2020-08-05 06:10:01 UTC
oVirt Team: Virt
Embargoed:
pm-rhel: ovirt-4.4?
pm-rhel: blocker?
michal.skrivanek: planning_ack?
pm-rhel: devel_ack+
pm-rhel: testing_ack+


Attachments (Terms of Use)
Changes for sysprep (37.44 KB, image/png)
2019-09-04 13:47 UTC, Steven Rosenberg
no flags Details
Current changes for Run Once with the Additional CD (32.81 KB, image/png)
2019-09-06 10:56 UTC, Steven Rosenberg
no flags Details
Current changes for Run Once Referencing only sysprep (37.44 KB, image/png)
2019-09-09 14:03 UTC, Steven Rosenberg
no flags Details
Current changes for Run Once Referencing only sysprep (33.36 KB, image/png)
2019-09-09 14:09 UTC, Steven Rosenberg
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 101798 0 'None' MERGED engine: Replace floppy support for Versions >= 4.4 2020-11-09 15:33:03 UTC
oVirt gerrit 102912 0 'None' MERGED engine: Replace floppy file support for Ver >= 4.4 2020-11-09 15:33:03 UTC
oVirt gerrit 108821 0 master MERGED core: OOB Experience requires Unattend.xml 2020-11-09 15:33:03 UTC

Description Michal Skrivanek 2016-02-11 11:34:28 UTC
we should change the drivers (and sysprep) delivery method to CDROM instead of the legacy floppy which is causing lots of other issues. We have vmpayload working via a second CDROM so it should not be a big deal...

+++ This bug was initially created as a clone of Bug #1300959 +++

[snip]

--- Additional comment from Cole Robinson on 2016-02-10 15:57:23 CET ---

(In reply to Vadim Rozenfeld from comment #10)
> (In reply to Cole Robinson from comment #9)
> > I assume it would mean using a second IDE CDROM device, that should work as
> > described here:
> > 
> > https://rwmj.wordpress.com/2015/04/07/tip-virt-install-windows-with-virtio-
> > device-drivers/
> > 
> > I originally thought the .vfd was only for compat with older windows (I
> > think pre windows 7) that couldn't get disk drivers from a floppy.
> 
> Technically, only WXp and WS2003 require floppy image for installing
> boot-time viostor (virtio-blk) driver. The rest of the drivers for all kinds
> of supported platforms were added in response to requests coming from
> different
> teams and people. In any case, having a second IDE CDROM is always good. But
> if don't have it in the system, then floppy image is the only way to provide
> virtio-win drivers while installing the OS.

Thanks for the info. Though I wonder in what _realistic_ will a windows 10 VM that is installing virtio drivers not be able to provide a second IDE CDROM?

What I'm trying to determine is if the requests to extend the .vfd images are actually a necessity for RHEV. Because it's going to continue to be an issue for every new windows release. Eventually we will run out of space again and need to use a new naming scheme for the vfd images which may require RHEV patches anyways for all I know.

It would be nice to have a plan to wean off of their usage regardless, or have RHEV do something akin to boxes which assembles the .vfd on the fly which is not much work, provided we alter the RPM to start distributing all the drivers in /usr/share.

But I guess the still unanswered bit is how exactly modern RHEV uses these images, if at all.

--- Additional comment from Michal Skrivanek on 2016-02-10 17:58:58 CET ---

second CDROM would be possible, it's just not there yet for historical reasons. Perhaps we can finally phase out floppy completely as we do not intend to support XP and 2003 anymore
it's too late for such change in 3.6, but it should be doable for next release. Is "vfd on the fly" a viable approach long-term?

--- Additional comment from Cole Robinson on 2016-02-10 18:06:47 CET ---

(In reply to Michal Skrivanek from comment #12)
> second CDROM would be possible, it's just not there yet for historical
> reasons. 
> Perhaps we can finally phase out floppy completely as we do not
> intend to support XP and 2003 anymore
> it's too late for such change in 3.6, but it should be doable for next
> release. 

Okay. So just to clarify, latest RHEV depends on the .vfd files to install virtio-blk drivers for all windows version installs?

> Is "vfd on the fly" a viable approach long-term?

If you're going to rework things, and are dropping support for winxp and win2003, I'd suggest looking into the second cdrom method instead.

But I think generating the vfd image is viable long term. The logic would basically be:

- Determine the windows version+arch being installed
- Find the associated drivers in /usr/share/virtio-win/*
- Run the correct mkisofs command to generate a .vfd

--- Additional comment from Michal Skrivanek on 2016-02-10 18:38:14 CET ---

(In reply to Cole Robinson from comment #13)
> Okay. So just to clarify, latest RHEV depends on the .vfd files to install
> virtio-blk drivers for all windows version installs?

yes. there is no other way currently
 
> > Is "vfd on the fly" a viable approach long-term?
> 
> If you're going to rework things, and are dropping support for winxp and
> win2003, I'd suggest looking into the second cdrom method instead.
>
> But I think generating the vfd image is viable long term. The logic would
> basically be:
> 
> - Determine the windows version+arch being installed
> - Find the associated drivers in /usr/share/virtio-win/*
> - Run the correct mkisofs command to generate a .vfd

we have similar code for both, mkisofs for sysprep floppy, 2nd CD for VM payload. 
I guess...if CD has all of them it saves the hassle to pick up only the right version

--- Additional comment from Cole Robinson on 2016-02-10 20:18:05 CET ---

(In reply to Michal Skrivanek from comment #14)
> 
> we have similar code for both, mkisofs for sysprep floppy, 2nd CD for VM
> payload. 
> I guess...if CD has all of them it saves the hassle to pick up only the
> right version

Oh, so you're already using a custom second CDROM?

FWIW I think boxes/libosinfo puts sysprep + first round of drivers on the generated vfd, then uses a generated iso for firstboot driver installation of drivers that aren't install time essential.

--- Additional comment from Michal Skrivanek on 2016-02-11 12:28:44 CET ---

well, we do have a concept of 2 CDROMs, it is used in special cases (so the code is not exercised too often) and it was a nightmare to get it working due to plenty historical assumptions around "one cdrom is enough for everyone"

But it should be feasible today. Another reason to get rid of floppy as our delivery method for drivers and sysprep is our plan to support q35 chipset. q35 doesn't have a floppy.

Let me open a follow bug about that and keep this one for figuring out what to do in 3.6 where we have to keep using vfd

Comment 1 Jiri Belka 2016-02-11 11:41:48 UTC
does windows installer automatically search on 2nd cdrom for drivers (like it does in case of floppy)?

Comment 2 Yaniv Kaul 2016-02-12 19:55:38 UTC
Unless there's not enough room in the floppy for all relevant drivers, I honestly don't see the use specifically for this. However, I do recall a request for multiple CD-ROMs support - something about need the OS in one, the application in another or something like that.
Worth re-reviewing the use of IDE vs. SATA or SCSI CD-ROM.

Comment 3 Shahar Havivi 2016-02-14 07:10:27 UTC
This will require a user action inside a guest (we can supply it via our user guide),
When creating a sysprep windows is looking in the first floppy drive for the sysprep.inf/Unattended.xml file unless it overridden via the registry key: HKLM\System\Setup!UnattendFile
There is no issue of room since the file its a small text file.

Comment 4 Lev Veyde 2016-02-17 08:55:25 UTC
(In reply to Yaniv Kaul from comment #2)
> Unless there's not enough room in the floppy for all relevant drivers, I
> honestly don't see the use specifically for this. However, I do recall a
> request for multiple CD-ROMs support - something about need the OS in one,
> the application in another or something like that.
> Worth re-reviewing the use of IDE vs. SATA or SCSI CD-ROM.

Or USB CD-ROM.

Comment 5 Yaniv Lavi 2016-02-24 07:58:43 UTC
Can we emulate a USB flash drive instead of a second CD? It sound more suitable to pass payloads or the driver.

Comment 6 Michal Skrivanek 2016-02-24 08:15:52 UTC
Yes we can. Though not sure we want to. Still investigating, we want to make it automatic if possible, to avoid registry modification

Comment 7 Yaniv Kaul 2016-02-24 08:43:44 UTC
I still prefer having it in the floppy (drop XP and 2003) if it fits.
However, if we have other needs for an additional CD-ROM - let's have one. I think USB is a bad idea (I wish we could drop support from USB devices for server VMs - don't see why they'd need this HW and it only has bugs, inc. security bugs and performance implications - at least used to be).

Comment 8 Yaniv Lavi 2016-03-23 08:57:14 UTC
I would push this to 4.0 since VDF drivers will be added 6.8.z and 7.2.z.

Comment 14 Sandro Bonazzola 2016-05-02 10:01:50 UTC
Moving from 4.0 alpha to 4.0 beta since 4.0 alpha has been already released and bug is not ON_QA.

Comment 15 Yaniv Lavi 2016-05-23 13:17:27 UTC
oVirt 4.0 beta has been released, moving to RC milestone.

Comment 16 Yaniv Lavi 2016-05-23 13:21:23 UTC
oVirt 4.0 beta has been released, moving to RC milestone.

Comment 17 Michal Skrivanek 2016-05-23 13:28:06 UTC
nothing is planned in 4.0 at this point on oVirt side. Drivers supposedly fit and one can use SATA on q35 which works

Comment 18 Tomas Jelinek 2016-08-11 07:33:51 UTC
Based on comment 17 lowering priority but we would still like to see this in 4.1

Comment 19 Tomas Jelinek 2016-12-14 14:17:29 UTC
will not make 4.1 due to capacity

Comment 20 Tomas Jelinek 2017-09-15 08:19:08 UTC
will not make 4.2 due to capacity

Comment 22 Greg Sheremeta 2018-08-08 18:29:40 UTC
I recently tried to install Windows in a new VM. I couldn't figure out how to use the floppy or the ISO. Old and new documentation was pointing me in different directions.

Which leads me to the question, can this be automagic?

Why does the user need to bother with all the intermediate steps to load the drivers? We know we want them to use the virtio driver for the hard drive. It should be as easy as possible / just show up as an option.

Comment 23 Cole Robinson 2018-08-08 23:12:19 UTC
(In reply to Greg Sheremeta from comment #22)
> I recently tried to install Windows in a new VM. I couldn't figure out how
> to use the floppy or the ISO. Old and new documentation was pointing me in
> different directions.
> 
> Which leads me to the question, can this be automagic?
> 
> Why does the user need to bother with all the intermediate steps to load the
> drivers? We know we want them to use the virtio driver for the hard drive.
> It should be as easy as possible / just show up as an option.

There's a bug tracking changing the directory structure which will help windows automatically find the drivers,  at least partly

https://bugzilla.redhat.com/show_bug.cgi?id=1223668

Comment 24 lijin 2018-09-12 03:30:18 UTC
Please be aware that virtio-win package has dropped floppy files from rhel8.0 according to bz1533540

Comment 25 Ryan Barry 2019-01-21 14:53:45 UTC
Re-targeting to 4.3.1 since it is missing a patch, an acked blocker flag, or both

Comment 26 Steven Rosenberg 2019-09-04 13:47:53 UTC
Created attachment 1611519 [details]
Changes for sysprep

Comment 27 Steven Rosenberg 2019-09-06 10:56:23 UTC
Created attachment 1612313 [details]
Current changes for Run Once with the Additional CD

Comment 28 Steven Rosenberg 2019-09-09 14:03:39 UTC
Created attachment 1613185 [details]
Current changes for Run Once Referencing only sysprep

Comment 29 Steven Rosenberg 2019-09-09 14:09:02 UTC
Created attachment 1613187 [details]
Current changes for Run Once Referencing only sysprep

Comment 30 Steven Rosenberg 2019-09-10 09:48:52 UTC
Note: Windows setup for installations requires the unattended file to be called Autounattend.xml for Windows Vista and later. For sealing the OS, the sysprep tool may still require the  unattend file to be called Unattend.xml for auto-detection, although the tool does support the "/unattend:<file_name>" command line option. 

Therefore validation and documentation should reflect these issues accordingly.

Comment 31 Marina Kalinin 2019-10-28 21:13:33 UTC
I wish I can tag this bug with "need knowledge transfer". 
Maybe I can - by adding a keyword.

Comment 40 RHEL Program Management 2020-05-04 11:36:03 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 43 Beni Pelled 2020-05-05 15:27:45 UTC
Update:
The problem seems to be somewhere around renaming the Sysprep file from Unattend.xml to Autounattend.xml,
Sysprep works when the filename is called Unattend.xml and doesn't work when the filename is Autounattend.xml - checked by the following steps:

1. Add the following line to the /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties (and restart ovirt-engine)
   os.windows_2012R2x64.sysprepFileName.value = Unattend.xml
2. Start a 'Windows 2012R2 X64' VM' (make sure the OS type is 'Windows 2012R2 X64')
3. On the VM, set the TZ to UTC London and run 'C:\Windows\System32\Sysprep\sysprep.exe /oobe /generalize /shutdown'
4. At this point, a sealing process is running - wait for the process to finish and the VM to shut down.
5. Re-run the VM as run-once and under 'Initial Run > Use Sysprep' set VM Hostname=WinTestSysprep and Time Zone=+2 Israel

Result:
- Hostname and TZ settings are injected as expected (WinTestSysprep, +2 Israel)
- Sysprep file is called Unattend.xml (under the CDROM drive)

The same flow without adding the sysprepFileName.value to the 00-defaults.properties cause the 
filename to be Autounattend.xml (as expected) and the settings are not injected.

In summary, the transition from Floppy to CDROM does work but right now it's with WA
- this issue can be tracked here or in a new bug.

Comment 44 Steven Rosenberg 2020-05-05 16:36:09 UTC
(In reply to Beni Pelled from comment #43)
> Update:
> The problem seems to be somewhere around renaming the Sysprep file from
> Unattend.xml to Autounattend.xml,
> Sysprep works when the filename is called Unattend.xml and doesn't work when
> the filename is Autounattend.xml - checked by the following steps:
> 
> 1. Add the following line to the
> /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties (and restart
> ovirt-engine)
>    os.windows_2012R2x64.sysprepFileName.value = Unattend.xml
> 2. Start a 'Windows 2012R2 X64' VM' (make sure the OS type is 'Windows
> 2012R2 X64')
> 3. On the VM, set the TZ to UTC London and run
> 'C:\Windows\System32\Sysprep\sysprep.exe /oobe /generalize /shutdown'
> 4. At this point, a sealing process is running - wait for the process to
> finish and the VM to shut down.
> 5. Re-run the VM as run-once and under 'Initial Run > Use Sysprep' set VM
> Hostname=WinTestSysprep and Time Zone=+2 Israel
> 
> Result:
> - Hostname and TZ settings are injected as expected (WinTestSysprep, +2
> Israel)
> - Sysprep file is called Unattend.xml (under the CDROM drive)
> 
> The same flow without adding the sysprepFileName.value to the
> 00-defaults.properties cause the 
> filename to be Autounattend.xml (as expected) and the settings are not
> injected.
> 
> In summary, the transition from Floppy to CDROM does work but right now it's
> with WA
> - this issue can be tracked here or in a new bug.

OK, but we were supposed to try it with the audit command [1] when we have the Autounattend.xml. Maybe we can try that. 

[1] C:\Windows\System32\Sysprep\sysprep.exe /audit /reboot /unattend:D:\Autounattend.xml

Comment 45 Ryan Barry 2020-05-05 16:37:52 UTC
We really need it to work automatically the same way it did in 4.3 rather than asking administrators to sysprep in a specific way. No need to check the audit command

Comment 46 Beni Pelled 2020-06-02 13:00:46 UTC
Verified with:
- ovirt-engine-4.4.1.1-0.5.el8ev.noarch
- libvirt-6.0.0-17.module+el8.2.0+6257+0d066c28.x86_64
- vdsm-4.40.18-1.el8ev.x86_64

Verification steps:
2. Start a 'Windows 2012R2 X64' VM'
3. On the VM, set the TZ to UTC London and run 'C:\Windows\System32\Sysprep\sysprep.exe /oobe /generalize /shutdown'
   (At this point, a sealing process is running - wait for the process to finish and the VM to shutdown)
4. Re-run the VM as run-once and under 'Initial Run > Use Sysprep' set VM Hostname=WinSysprep and Time Zone=+2 Israel

Result:
- VM is up and running with HOSTNAME=WinSysprep and TZ=Israel

Comment 47 Sandro Bonazzola 2020-08-05 06:10:01 UTC
This bugzilla is included in oVirt 4.4.0 release, published on May 20th 2020.

Since the problem described in this bug report should be
resolved in oVirt 4.4.0 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.