Red Hat Bugzilla – Bug 1024833
[RHEV-M] Existing Pooled Windows 7 VMs are constantly prompted for a reboot to apply changes
Last modified: 2013-11-19 05:08:57 EST
Description of problem:
Whenever a VM is restored from a snapshot (Manually on a Peristsant or automaticaly from the User portal on Pooled VM) the system re-enumerates the drivers and prompts for a reboot.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
We tried recreating the Templates and Pools. We did this with both 3.1.14 and the new 3.2.15 Redhat Tool sets. We are still getting the driver enumeration issue and repeating reboot prompts.
As mentioned previously snapshots restores seem to be triggering the driver re-enumeration issue. This happens both on Persistent VMs and Pooled VMs.
This is how to reproduce the problem:
1. Update Persistent VM with 3.1.14 or 3.2.15 redhat Tools
2. Create Template
3. Create a pool
4. Power up VMS from Admin portal and let them finish syspreping then shut them
5. Power up a VM from the pool from the User Portal
Note: On the initial Power up of a new Vm from the user portal you will
not receive a reboot prompt.
6. Shutdown that vm
7. Power up the same VM form the user portal. The drivers will reinstall and
you will be promoted to reboot. This will now happen EVERY time this VM is
started from now on.
A 64 bit windows 7 image with 3.1.14 tools shows the same behaviour
Peter, since it is reproducing in your environment, can you please compare the VM devices when running the VM for the first time as stateless (before shuting down the VM when it ran as stateless) and when running the VM for the second time as stateless?
In particular I'm interested to see if the sound device was duplicated in the process (but also if any other device was duplicated) because that will explain why the addresses of several devices changed for the customer (and this bug was already fixed so it will also explain why this bug is not reproducing in my environment)
Created attachment 821190 [details]
List of devices and drivers from the guest perspective
I'm attaching a gzipped tar archive containing 3 files. It is named Devices.tgz. There are two versions of the device list as seen from Windows 7 and a png screenshot of the installed drivers and reboot request.
The two device lists are the same. I just provided two different formats; a tab delimited csv and an ods. The device list was produced using a free tool called DevManView http://www.nirsoft.net/utils/device_manager_view.html.
I believe that the device list includes all the historic devices since the VM guest was created. The old ones are not connected. Only the most recent ones shows as connected.
I had manually disabled a number of sound devices earlier, but the later ones show as enabled.
I will create a fresh VM guest with a clean test a provide additional information. I'm sure the included files will provide helpful information.
(In reply to Peter Manzella from comment #6)
Thank you Peter.
> Created attachment 821190 [details]
> List of devices and drivers from the guest perspective
> I'm attaching a gzipped tar archive containing 3 files. It is named
> Devices.tgz. There are two versions of the device list as seen from Windows
> 7 and a png screenshot of the installed drivers and reboot request.
> The two device lists are the same. I just provided two different formats; a
> tab delimited csv and an ods. The device list was produced using a free tool
> called DevManView http://www.nirsoft.net/utils/device_manager_view.html.
> I believe that the device list includes all the historic devices since the
> VM guest was created. The old ones are not connected. Only the most recent
> ones shows as connected.
> I had manually disabled a number of sound devices earlier, but the later
> ones show as enabled.
IIUC the fact that there are many High Definition Audio Controller & High Definition Audio Device devices might mean that our suspicions were correct and the sound device was duplicated in each run.
> I will create a fresh VM guest with a clean test a provide additional
> information. I'm sure the included files will provide helpful information.
When you test it with a fresh VM guest, please check the devices in our database as well, because it is more useful for us ( by: select device, type, is_managed, address from vm_device where vm_id in (select vm_guid from vm_static where vm_name ilike '%windows7%'); )
Summary of my findings so far:
1. I couldn't reproduce it in my environment. I followed the reproduction steps and it seems that the virtio-serial and the rest of the devices still exist, no dialog is shown for devices that were reinstalled or request for reboot.
2. I thought it might be related to bug that was already fixed which was about the sound device being duplicated in the process of restarting stateless VM. so I did similar thing: when the VM stopped, I defined new network interface for it, and when I started the VM the addresses of the balloon device and the virtio-serial were changed.
In my guest it seems that the new devices weren't installed automatically so I wasn't asked for reboot, but in the device manager I saw they are marked as devices without proper driver..
3. If the sound device was duplicated, it will explain the reported results:
- The audio related devices were reinstalled because new sound device was added
- The addresses of the balloon device and of the virtio-serial device were changed, thus they were detected as new devices and their drivers were reinstalled
4. The sound device being duplicated was solved (bz 1004066). If it was indeed duplicated in this bug, then that fix should solve this bug as well. I'm waiting for information from Peter about how the devices look like in our DB in such flow to confirm that the sound device was duplicated.
5. Even if this bug will be solved by the solution for bz 1004066, we still have an open issue: vdsm doesn't send the addresses of balloon and virtio-serial devices to libvirt. It means that if we stop the VM, add a device for that VM and then start it again, the addresses of the balloon and virtio-serial might change (like they were changed when I added nic). In order to solve this we need:
- change the virtio-serial to be managed device (so that it won't be removed when restoring configuration from snapshot)
- pass the addresses of the balloon and virtio-serial devices from vdsm to libvirt
Created attachment 821325 [details]
List of devices and drivers from the guest and DB perspective after fresh install
Here is the requested data in the attached tar archive. I did a fresh install of Windows 7. I installed the tools from RHEV-toolsSetup_3.2_15.iso. Each of the included files is preceded by an identifying number as follows.
01 - After the installation, but before stateless is enabled.
02 - From the first cold boot after stateless is enabled.
No new drivers installed.
03 - From the second cold boot after stateless enabled.
04 - From the restart after the second cold boot to finish driver install.
05 - From the third cold boot after stateless enabled.
06 - From the restart after the third cold boot to finish driver install.
I started testing before reading the last updates, so there are only DB query results for 04, 05, and 06. I think they show additional sound devices created.
The csv files are tab delimited.
after investigation we think bug 1004066 will indeed be good enough. 3.2.z clone (bug 1015134) has a hotfix approved already.
The issue that virtio-serial and balloon are not managed devices should not demonstrate anymore, however it should be fixed, I've opened bug 1028387, and I propose to close duplicate this one to bug 1004066
Initial testing shows that the issue is no longer present, but, of course, the follow-on bugs should be fixed so that other devices don't stimulate the issue in the future. Thanks.
*** This bug has been marked as a duplicate of bug 1004066 ***