Bug 692634

Summary:	VM being installed win7 can't reboot after files are copied to the disk using virtio-win 1.1.20
Product:	Red Hat Enterprise Linux 6	Reporter:	Lucas Meneghel Rodrigues <lmr>
Component:	virtio-win	Assignee:	Vadim Rozenfeld <vrozenfe>
Status:	CLOSED DUPLICATE	QA Contact:	Virtualization Bugs <virt-bugs>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	6.1	CC:	areis, qzhang, tburke
Target Milestone:	rc
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2011-04-02 02:07:28 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Lucas Meneghel Rodrigues 2011-03-31 18:39:19 UTC

Description of problem: Found a problem on RHEL 6.1 daily testing using virtio-win drivers v 1.1.20, straight from the release page:

http://download.lab.bos.redhat.com/devel/RHEV/virtio-win/

The setup stage copies all files to the win partition, all good, see screenshot:

http://autotest.virt.bos.redhat.com/results/586-autotest/virtlab5.virt.bos.redhat.com/kvm.rhel61-brew.hugepages.virtio_blk.smp2.Win7.64.virtio_net.unattended_install.cdrom/debug/screendumps_vm1/vm1_2011-03-28_06-13-14.jpg

Then the VM gets stuck right after that step:

http://autotest.virt.bos.redhat.com/results/586-autotest/virtlab5.virt.bos.redhat.com/kvm.rhel61-brew.hugepages.virtio_blk.smp2.Win7.64.virtio_net.unattended_install.cdrom/debug/screendumps_vm1/vm1_2011-03-28_06-13-19.jpg

And it'll sat on that screen until the test timeout is reached. See screenshot from the same VM almost 3:30 hours later:

http://autotest.virt.bos.redhat.com/results/586-autotest/virtlab5.virt.bos.redhat.com/kvm.rhel61-brew.hugepages.virtio_blk.smp2.Win7.64.virtio_net.unattended_install.cdrom/debug/screendumps_vm1/vm1_2011-03-28_09-58-42.jpg

Version-Release number of selected component (if applicable): 1.1.20

How reproducible: Close to 100% (it's happening on every daily sanity test job for RHEL 6.1)

Steps to Reproduce:
1. Start a KVM VM from RHEL 6.1 as you can see it here:

http://autotest.virt.bos.redhat.com/results/586-autotest/virtlab5.virt.bos.redhat.com/kvm.rhel61-brew.hugepages.virtio_blk.smp2.Win7.64.virtio_net.unattended_install.cdrom/debug/kvm.rhel61-brew.hugepages.virtio_blk.smp2.Win7.64.virtio_net.unattended_install.cdrom.DEBUG

The command used was:

/usr/local/autotest/tests/kvm/qemu -name 'vm1' -monitor unix:'/tmp/monitor-humanmonitor1-20110328-002357-e75i',server,nowait -qmp unix:'/tmp/monitor-qmpmonitor1-20110328-002357-e75i',server,nowait -serial unix:'/tmp/serial-20110328-002357-e75i',server,nowait -drive file='/tmp/kvm_autotest_root/images/win7-64.qcow2',index=0,if=virtio,cache=none -device virtio-net-pci,netdev=idu0ahDz,mac='9a:da:f6:00:69:33',id='idHukjgf' -netdev tap,id=idu0ahDz,ifname='t0-002357-e75i',script='/usr/local/autotest/tests/kvm/scripts/qemu-ifup',downscript='no' -m 2048 -smp 2 -drive file='/tmp/kvm_autotest_root/isos/windows/en_windows_7_ultimate_x64_dvd_x15-65922.iso',media=cdrom,index=1 -drive file='/tmp/kvm_autotest_root/isos/windows/winutils.iso',media=cdrom,index=2 -drive file='/tmp/kvm_autotest_root/isos/virtio-win.iso',media=cdrom,index=3 -fda '/tmp/kvm_autotest_root/images/win7-64/answer.vfd' -vnc :0  -boot d -mem-path /mnt/kvm_hugepage

The VM was started with a windows answer file present on a virtual floppy, that contains paths to the virtio drivers on a CD. The full answer file can be seen on the DEBUG log referred above:

03/28 05:58:38 DEBUG|test_setup:0352| 		<component name="Microsoft-Windows-PnpCustomizationsWinPE"
03/28 05:58:38 DEBUG|test_setup:0352| 			processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35"
03/28 05:58:38 DEBUG|test_setup:0352| 			language="neutral" versionScope="nonSxS"
03/28 05:58:38 DEBUG|test_setup:0352| 			xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State"
03/28 05:58:38 DEBUG|test_setup:0352| 			xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
03/28 05:58:38 DEBUG|test_setup:0352| 			<DriverPaths>
03/28 05:58:38 DEBUG|test_setup:0352| 				<PathAndCredentials wcm:keyValue="1" wcm:action="add">
03/28 05:58:38 DEBUG|test_setup:0352| 					<Path>F:\viostor\Win7\amd64</Path>
03/28 05:58:38 DEBUG|test_setup:0352| 				</PathAndCredentials>
03/28 05:58:38 DEBUG|test_setup:0352| 				<PathAndCredentials wcm:keyValue="2" wcm:action="add">
03/28 05:58:38 DEBUG|test_setup:0352| 					<Path>F:\NetKVM\Vista\amd64</Path>
03/28 05:58:38 DEBUG|test_setup:0352| 				</PathAndCredentials>
03/28 05:58:38 DEBUG|test_setup:0352| 			</DriverPaths>

The drivers are on a CD, that was built with the contents of the 1.1.20 release, from the page refered above.

2. Watch windows install process, until it finishes copying the files
3. When it gets to reboot the vm, it will get stuck
  
Actual results: VM gets stuck

Expected results: VM reboots, and windows setup ends successfuly.

Additional info: All the debug logs, VM screenshots and everything else can be found here:

http://autotest.virt.bos.redhat.com/results/586-autotest/virtlab5.virt.bos.redhat.com/kvm.rhel61-brew.hugepages.virtio_blk.smp2.Win7.64.virtio_net.unattended_install.cdrom/debug/

In the same job, we can get the exact versions of qemu-kvm and kernel being used:

03/28 00:23:53 DEBUG|base_utils:0106| [stdout] Installed:
03/28 00:23:53 DEBUG|base_utils:0106| [stdout]   gpxe-roms-qemu.noarch 0:0.9.7-6.7.el6                                         
03/28 00:23:53 DEBUG|base_utils:0106| [stdout]   qemu-img.x86_64 2:0.12.1.2-2.152.el6                                          
03/28 00:23:53 DEBUG|base_utils:0106| [stdout]   qemu-kvm.x86_64 2:0.12.1.2-2.152.el6                                          
03/28 00:23:53 DEBUG|base_utils:0106| [stdout]   qemu-kvm-tools.x86_64 2:0.12.1.2-2.152.el6                                    
03/28 00:23:53 DEBUG|base_utils:0106| [stdout]   seabios.x86_64 0:0.6.1.2-3.el6                                                
03/28 00:23:53 DEBUG|base_utils:0106| [stdout]   spice-server.x86_64 0:0.8.0-1.el6                                             
03/28 00:23:53 DEBUG|base_utils:0106| [stdout]   vgabios.noarch 0:0.6b-3.5.el6  

kernel:

03/28 00:20:22 DEBUG|base_utils:0074| Running 'rpm -qp /usr/local/autotest/tmp/kernel-2.6.32-125.el6.x86_64.rpm'

Comment 2 Lucas Meneghel Rodrigues 2011-04-01 00:19:38 UTC

Update: Since I've learned there's a new repo for virtio drivers, I have updated the testing infrastructure to use the latest drivers available as of today (March 31st 2011):

http://download.devel.redhat.com/brewroot/packages/virtio-win-prewhql/0.1/9/win/virtio-win-prewhql-0.1.zip

I am running a RHEL 6.1 sanity test with this version (0.1-9), and the problem is still happening.

Comment 3 Qunfang Zhang 2011-04-01 02:32:33 UTC

Hi, Lucas Meneghel Rodrigues

Which kernel is installed on your host?  I also met this problem when rebooting windows guest. And have filed a qemu-kvm bug:
Bug 684719 - Windows guests hang when rebooting with kernel-2.6.32-121.el6

Thanks

Comment 4 Lucas Meneghel Rodrigues 2011-04-01 02:42:48 UTC

I have described the host kernel version on the description:

kernel-2.6.32-125.el6.x86_64.rpm

Well, I haven't had such problems with ide drives, which is the condition you are mentioning on bug 684719... Wonder if it's really the same problem.

Comment 5 Lucas Meneghel Rodrigues 2011-04-01 19:50:39 UTC

Hi Qunfang:

Ok, I have made some more tests:

* The problem does not happen on RHEL 5.X or RHEL 6.0
* I created a special job to verify this bug under 6.1 - I made autotest perform installs of windows with ide and rtl8139 as block layer and network card, respectively. The problem happens on this combination as well.

So, breaking my initial assumption that the problem had to do with the virtio drivers. Based on this observation, the problem is not related to the virtio drivers, rather, it is a problem in some of the components of RHEL 6.1 we are using. The components we are using are:

kernel: 2.6.32-128.el6.x86_64
Userspace:
04/01 08:47:58 DEBUG|base_utils:0106| [stdout]   gpxe-roms-qemu.noarch 0:0.9.7-6.7.el6                                         
04/01 08:47:58 DEBUG|base_utils:0106| [stdout]   qemu-img.x86_64 2:0.12.1.2-2.153.el6                                          
04/01 08:47:58 DEBUG|base_utils:0106| [stdout]   qemu-kvm.x86_64 2:0.12.1.2-2.153.el6                                          
04/01 08:47:58 DEBUG|base_utils:0106| [stdout]   qemu-kvm-tools.x86_64 2:0.12.1.2-2.153.el6                                    
04/01 08:47:58 DEBUG|base_utils:0106| [stdout]   seabios.x86_64 0:0.6.1.2-3.el6                                                
04/01 08:47:58 DEBUG|base_utils:0106| [stdout]   spice-server.x86_64 0:0.8.0-1.el6                                             
04/01 08:47:58 DEBUG|base_utils:0106| [stdout]   vgabios.noarch 0:0.6b-3.5.el6  

So, this indeed seems to be a duplicate of Bug 684719... whomever is handling this bug, you can close this as a duplicate of that bug.

Comment 6 Qunfang Zhang 2011-04-02 02:06:37 UTC

Hi, Lucas
Thanks for the testing and if the problem blocks you, maybe you can try kernel-120. I am using kernel-120 as well to avoid the reboot issue to go ahead with my test currently.
Then I will close it. 

Best regards,
Qunfang

Comment 7 Qunfang Zhang 2011-04-02 02:07:28 UTC


*** This bug has been marked as a duplicate of bug 684719 ***