Bug 548197

Summary:	Boot failed when rebooting Windows guest with virtio-blk device
Product:	Red Hat Enterprise Linux 5	Reporter:	James Ko <koj>
Component:	kvm	Assignee:	Vadim Rozenfeld <vrozenfe>
Status:	CLOSED NOTABUG	QA Contact:	Virtualization Bugs <virt-bugs>
Severity:	medium	Docs Contact:
Priority:	high
Version:	5.4.z	CC:	cpelland, gleb, koj, llim, tburke, virt-maint, ykaul
Target Milestone:	rc
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2010-04-13 08:35:06 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	580948

Description James Ko 2009-12-16 21:20:24 UTC

Description of problem:
Rebooting Windows guest with virtio-block disk configured does not succeed.

Version-Release number of selected component (if applicable):
5.4.9


How reproducible:
everytime.


Steps to Reproduce:
1. Start qemu-kvm -drive boot=on,if=virtio,file=virtio-disk.qcow2 -boot c -cdrom W2K8R2.iso -fda virtio-drivers.vfd ... 

2. Press F12 for boot menu
3. Select boot from cd-rom to start installation of Windows guest
4. Install virtio-block drivers from A: when selecting installation drive
5. Continue guest installation
6. When guest reboots during install subsequent boots from disk fail
7. killing qemu-kvm process and restarting with same arguments the boot is successful

Or with guest already running with virtio-block disk.
1. reboot Windows from within the guest
  
Actual results:
Booting from Hard Disk...
Boot failed: could not read the boot disk

FATAL: No bootable device.


Expected results:
Normal reboot from Hard Disk of Windows.


Additional info:

Comment 1 Dor Laor 2010-01-20 09:34:45 UTC

What is the virtio-win version number?

Comment 2 James Ko 2010-01-20 10:42:52 UTC

(In reply to comment #1)
> What is the virtio-win version number?    

virtio-win-1.0.0-7.39539.el5.noarch.rpm

Comment 3 James Ko 2010-01-20 10:45:41 UTC

FYI.. Windows guest is Server 2008 R2 if it wasn't obvious from the -cdrom option.

Comment 6 Vadim Rozenfeld 2010-03-09 22:46:58 UTC

sorry about being slow to respond. I tried to reproduce this problem a number of times without success. The configuration is as follow:

uname -r
2.6.18-164.el5

(qemu) info version
0.9.1

QEMU command line:
./qemu-system-x86_64 -m 512 -smp 1 -net nic,vlan=0,macaddr=00:1a:4a:01:00:bf,model=rtl8139 -net user -monitor stdio -drive boot=on,if=virtio,file=/home/vadimr/work/images/w2k8r2.qcow2 -boot c -cdrom /home/vadimr/work/install/en_windows_server_2008_r2_standard_enterprise_datacenter_and_web_x64_dvd_x15-59754.iso -fda /home/vadimr/work/install/virtio-win-1.0.0-7.39539.el5/virtio-drivers-1.0.0.vfd 

Guest:

Host Name:                 WIN-53Q8GD1FVSL

OS Name:                   Microsoft Windows Server 2008 R2 Standard 

OS Version:                6.1.7600 N/A Build 7600

OS Manufacturer:           Microsoft Corporation

OS Configuration:          Standalone Server

OS Build Type:             Multiprocessor Free

Registered Owner:          Windows User

Registered Organization:   

Product ID:                00477-001-0000421-84735

Original Install Date:     3/9/2010, 12:53:49 PM

System Boot Time:          3/9/2010, 1:42:53 PM

System Manufacturer:       Red Hat

System Model:              KVM

System Type:               x64-based PC

Processor(s):              1 Processor(s) Installed.

                           [01]: Intel64 Family 6 Model 6 Stepping 3 GenuineIntel ~2992 Mhz

BIOS Version:              QEMU QEMU, 1/1/2007

Windows Directory:         C:\Windows

System Directory:          C:\Windows\system32

Boot Device:               \Device\HarddiskVolume1

System Locale:             en-us;English (United States)

Input Locale:              en-us;English (United States)

Time Zone:                 (UTC-08:00) Pacific Time (US & Canada)

Total Physical Memory:     512 MB

Available Physical Memory: 188 MB

Virtual Memory: Max Size:  1,536 MB

Virtual Memory: Available: 1,170 MB

Virtual Memory: In Use:    366 MB

Page File Location(s):     C:\pagefile.sys

Domain:                    WORKGROUP

Logon Server:              \\WIN-53Q8GD1FVSL

Hotfix(s):                 N/A

Network Card(s):           1 NIC(s) Installed.

                           [01]: Realtek RTL8139C+ Fast Ethernet NIC

                                 Connection Name: Local Area Connection

                                 DHCP Enabled:    Yes

                                 DHCP Server:     10.0.2.2

                                 IP address(es)

                                 [01]: 10.0.2.15

                                 [02]: fe80::f04e:2466:b88e:ce8a



James, can you please post the same information from your system?
Would it be possible to try installing Win2K3 32/64, to check whether you get the same error or not?

Thanks,
Vadim.

Comment 7 James Ko 2010-04-06 19:13:04 UTC

debugshell# uname -r
2.6.23waas64.

qemu version is 0.9.1

Full command line...
/sw/kvm/bin/qemu -pidfile /var/run/vb1.pid -daemonize -boot c -m 512 -option-rom /sw/kvm/share/qemu/pxe-rtl8139.bin -name "win2k8" -cdrom /local/local1/vbs/W2K8R2.iso -fda /local/local1/vbs/virtio-drivers.vfd -usbdevice tablet -serial telnet:localhost:2001,server,nowait,nodelay -cpu qemu64 -no-hpet -rtc-td-hack -vnc :1 -net nic,vlan=0,macaddr=00:21:D8:AB:D2:A3,model=rtl8139 -net tap,vlan=0,ifname=vblade10,script=/tmp/vblade10,downscript=/sw/kvm/bin/qemu-ifdown -drive boot=on,index=0,if=virtio,file=/vbspace/vb1-disk0,cache=off 

C:\Users\Administrator>systeminfo

Host Name:                 WIN-NNVRDQO8L3O
OS Name:                   Microsoft Windows Server 2008 R2 Standard
OS Version:                6.1.7600 N/A Build 7600
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Server
OS Build Type:             Multiprocessor Free
Registered Owner:          Windows User
Registered Organization:
Product ID:                00477-001-0000421-84477
Original Install Date:     4/6/2010, 1:04:29 PM
System Boot Time:          4/6/2010, 1:03:37 PM
System Manufacturer:       Red Hat
System Model:              KVM
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: Intel64 Family 6 Model 6 Stepping 3 GenuineIntel ~1995 Mhz
BIOS Version:              QEMU QEMU, 1/1/2007
Windows Directory:         C:\Windows
System Directory:          C:\Windows\system32
Boot Device:               \Device\HarddiskVolume1
System Locale:             en-us;English (United States)
Input Locale:              en-us;English (United States)
Time Zone:                 (UTC-08:00) Pacific Time (US & Canada)
Total Physical Memory:     512 MB
Available Physical Memory: 142 MB
Virtual Memory: Max Size:  1,536 MB
Virtual Memory: Available: 1,152 MB
Virtual Memory: In Use:    384 MB
Page File Location(s):     C:\pagefile.sys
Domain:                    WORKGROUP
Logon Server:              \\WIN-NNVRDQO8L3O
Hotfix(s):                 N/A
Network Card(s):           1 NIC(s) Installed.
                           [01]: Realtek RTL8139C+ Fast Ethernet NIC
                                 Connection Name: Local Area Connection
                                 DHCP Enabled:    Yes
                                 DHCP Server:     255.255.255.255
                                 IP address(es)
                                 [01]: 169.254.175.115
                                 [02]: fe80::5cf3:d2d9:67f9:af73

Comment 8 James Ko 2010-04-06 19:38:33 UTC

This  was also repeatable using Windows 2003 and virtio block drivers 41879 for x86.

qemu monitor system_reset is not enough to successfully boot from the disk
qemu process with guest must be completely stopped and restarted before install continues

Comment 9 James Ko 2010-04-06 20:19:38 UTC

A little more information...

Not loading the option-rom did NOT make a difference.

Reproducible on normal Windows guest initiated restart as well.
Does not happen just during install.

Comment 10 James Ko 2010-04-06 20:48:37 UTC

Possible problem may be that the boot=on flag isn't set for the disk after initial boot which is required for booting virtio disk.

Comment 11 Vadim Rozenfeld 2010-04-07 10:19:38 UTC

(In reply to comment #9)
> A little more information...
> 
> Not loading the option-rom did NOT make a difference.
> 
> Reproducible on normal Windows guest initiated restart as well.
> Does not happen just during install.    

Normal means IDE?

Comment 12 James Ko 2010-04-07 17:18:11 UTC

(In reply to comment #11)
> (In reply to comment #9)
> > A little more information...
> > 
> > Not loading the option-rom did NOT make a difference.
> > 
> > Reproducible on normal Windows guest initiated restart as well.
> > Does not happen just during install.    
> 
> Normal means IDE?    

Means restart from within Windows (Start->Shutdown->Restart).

Comment 13 Vadim Rozenfeld 2010-04-07 20:12:42 UTC

(In reply to comment #12)
> (In reply to comment #11)
> > (In reply to comment #9)
> > > A little more information...
> > > 
> > > Not loading the option-rom did NOT make a difference.
> > > 
> > > Reproducible on normal Windows guest initiated restart as well.
> > > Does not happen just during install.    
> > 
> > Normal means IDE?    
> 
> Means restart from within Windows (Start->Shutdown->Restart).    

Just trying to narrow down the problem. 
Could you re-check it with if=ide instead of virtio?
Thanks,
Vadim

Comment 14 James Ko 2010-04-07 20:47:26 UTC

> 
> Just trying to narrow down the problem. 
> Could you re-check it with if=ide instead of virtio?
> Thanks,
> Vadim    

Definitely not a problem with ide device.  I normally use that and was testing with virtio.
I notice the IDE does not require boot=on flag.

Comment 15 Vadim Rozenfeld 2010-04-08 04:43:38 UTC

(In reply to comment #14)
> > 
> > Just trying to narrow down the problem. 
> > Could you re-check it with if=ide instead of virtio?
> > Thanks,
> > Vadim    
> 
> Definitely not a problem with ide device.  I normally use that and was testing
> with virtio.
> I notice the IDE does not require boot=on flag.    

boot=on flag means using extboot.bin optional ROM. IDE does require this flag
for booting guest with extended boot. 
Can guest be started with if=ide,boot=on ?

Comment 16 Dor Laor 2010-04-08 12:35:23 UTC

IDE should work w/ and w/o boot=on. Vadim asks you to test with IDE + boot=on to see if it is buggy too. It might be a bios issue.
Vadim, James what are you qemu specific versions (I mean rhel rpm numbers)?

Comment 17 Vadim Rozenfeld 2010-04-08 12:50:54 UTC

I my case it wasn't rpm. I build qemu by myself from git.engineering sources.
And yes, it sounds like extboot.bin is misplaced or simply missing.

Comment 18 James Ko 2010-04-09 18:34:24 UTC

with IDE and boot=on the reboot was successful

-drive boot=on,index=0,if=ide,file=/vbspace/vb1-disk0,cache=off

Comment 19 Vadim Rozenfeld 2010-04-09 18:59:58 UTC

Can you post the output from qemu monitor "info pci" command?

Comment 20 Vadim Rozenfeld 2010-04-09 19:05:27 UTC

(In reply to comment #19)
> Can you post the output from qemu monitor "info pci" command?    
when running with virtio device (sorry, forgot to mention it)

Comment 21 James Ko 2010-04-09 19:37:46 UTC

Output of "info pci" before and after reboot...

Initial boot: info pci
  Bus  0, device   0, function 0:
    Host bridge: PCI device 8086:1237
  Bus  0, device   1, function 0:
    ISA bridge: PCI device 8086:7000
  Bus  0, device   1, function 1:
    IDE controller: PCI device 8086:7010
      BAR4: I/O at 0xc000 [0xc00f].
  Bus  0, device   1, function 2:
    USB controller: PCI device 8086:7020
      IRQ 10.
      BAR4: I/O at 0xc020 [0xc03f].
  Bus  0, device   1, function 3:
    Bridge: PCI device 8086:7113
      IRQ 9.
  Bus  0, device   2, function 0:
    VGA controller: PCI device 1013:00b8
      BAR0: 32 bit memory at 0xc2000000 [0xc3ffffff].
      BAR1: 32 bit memory at 0xc4000000 [0xc4000fff].
  Bus  0, device   3, function 0:
    Ethernet controller: PCI device 10ec:8139
      IRQ 11.
      BAR0: I/O at 0xc100 [0xc1ff].
      BAR1: 32 bit memory at 0xffffffff [0x000000fe].
  Bus  0, device   4, function 0:
    SCSI controller: PCI device 1af4:1001
      IRQ 10.
      BAR0: I/O at 0xc200 [0xc23f].
  Bus  0, device   5, function 0:
    RAM controller: PCI device 1af4:1002
      IRQ 10.
      BAR0: I/O at 0xc240 [0xc25f].


After reboot: info pci
  Bus  0, device   0, function 0:
    Host bridge: PCI device 8086:1237
  Bus  0, device   1, function 0:
    ISA bridge: PCI device 8086:7000
  Bus  0, device   1, function 1:
    IDE controller: PCI device 8086:7010
      BAR4: I/O at 0xc000 [0xc00f].
  Bus  0, device   1, function 2:
    USB controller: PCI device 8086:7020
      IRQ 11.
      BAR4: I/O at 0xc020 [0xc03f].
  Bus  0, device   1, function 3:
    Bridge: PCI device 8086:7113
      IRQ 9.
  Bus  0, device   2, function 0:
    VGA controller: PCI device 1013:00b8
      BAR0: 32 bit memory at 0xc2000000 [0xc3ffffff].
      BAR1: 32 bit memory at 0xc4000000 [0xc4000fff].
  Bus  0, device   3, function 0:
    Ethernet controller: PCI device 10ec:8139
      IRQ 11.
      BAR0: I/O at 0xc100 [0xc1ff].
      BAR1: 32 bit memory at 0xc4001000 [0xc40010ff].
  Bus  0, device   4, function 0:
    SCSI controller: PCI device 1af4:1001
      IRQ 11.
      BAR0: I/O at 0xc200 [0xc23f].
  Bus  0, device   5, function 0:
    RAM controller: PCI device 1af4:1002
      IRQ 10.
      BAR0: I/O at 0xc240 [0xc25f].

Comment 22 Vadim Rozenfeld 2010-04-09 20:17:24 UTC

Thanks.
And now, could you please do the same, but with balloon device disabled
(RAM controller: PCI device 1af4:1002). You need "-balloon none" or something like this to be added to command line.

Comment 23 James Ko 2010-04-09 20:59:30 UTC

With -balloon none on the command line..

Before:
info pci
  Bus  0, device   0, function 0:
    Host bridge: PCI device 8086:1237
  Bus  0, device   1, function 0:
    ISA bridge: PCI device 8086:7000
  Bus  0, device   1, function 1:
    IDE controller: PCI device 8086:7010
      BAR4: I/O at 0xc000 [0xc00f].
  Bus  0, device   1, function 2:
    USB controller: PCI device 8086:7020
      IRQ 10.
      BAR4: I/O at 0xc020 [0xc03f].
  Bus  0, device   1, function 3:
    Bridge: PCI device 8086:7113
      IRQ 9.
  Bus  0, device   2, function 0:
    VGA controller: PCI device 1013:00b8
      BAR0: 32 bit memory at 0xc2000000 [0xc3ffffff].
      BAR1: 32 bit memory at 0xc4000000 [0xc4000fff].
  Bus  0, device   3, function 0:
    Ethernet controller: PCI device 10ec:8139
      IRQ 11.
      BAR0: I/O at 0xc100 [0xc1ff].
      BAR1: 32 bit memory at 0xffffffff [0x000000fe].
  Bus  0, device   4, function 0:
    SCSI controller: PCI device 1af4:1001
      IRQ 10.
      BAR0: I/O at 0xc200 [0xc23f].

After:
info pci
  Bus  0, device   0, function 0:
    Host bridge: PCI device 8086:1237
  Bus  0, device   1, function 0:
    ISA bridge: PCI device 8086:7000
  Bus  0, device   1, function 1:
    IDE controller: PCI device 8086:7010
      BAR4: I/O at 0xc000 [0xc00f].
  Bus  0, device   1, function 2:
    USB controller: PCI device 8086:7020
      IRQ 11.
      BAR4: I/O at 0xc020 [0xc03f].
  Bus  0, device   1, function 3:
    Bridge: PCI device 8086:7113
      IRQ 9.
  Bus  0, device   2, function 0:
    VGA controller: PCI device 1013:00b8
      BAR0: 32 bit memory at 0xc2000000 [0xc3ffffff].
      BAR1: 32 bit memory at 0xc4000000 [0xc4000fff].
  Bus  0, device   3, function 0:
    Ethernet controller: PCI device 10ec:8139
      IRQ 11.
      BAR0: I/O at 0xc100 [0xc1ff].
      BAR1: 32 bit memory at 0xc4001000 [0xc40010ff].
  Bus  0, device   4, function 0:
    SCSI controller: PCI device 1af4:1001
      IRQ 11.
      BAR0: I/O at 0xc200 [0xc23f].

Comment 24 James Ko 2010-04-09 23:33:20 UTC

Oh... "-balloon none" did not change the results of the reboot.  Still fails.

Comment 25 Dor Laor 2010-04-11 12:06:25 UTC

Vadim, can you post the exact steps for reproducing it that do work on your setup?

Comment 26 Vadim Rozenfeld 2010-04-11 12:22:02 UTC

I'm installing rhel5.4 right now to start my tests from beginning.
BTW, do you have any idea why after restart USB and SCSI controllers changed their
IRQ lines?

Comment 28 Dor Laor 2010-04-11 12:29:52 UTC

(In reply to comment #26)
> I'm installing rhel5.4 right now to start my tests from beginning.
> BTW, do you have any idea why after restart USB and SCSI controllers changed
> their
> IRQ lines?    

It might be the OS who remap it.
James, can you also test without index=0 ?

Comment 29 Vadim Rozenfeld 2010-04-12 10:03:46 UTC

(In reply to comment #25)
> Vadim, can you post the exact steps for reproducing it that do work on your
> setup?    

I checked it again on my freshly installed and updated rhel5.4 system. Both, system_reset and normal reset work fine for me.

[root@localhost libexec]# ./qemu-kvm -m 512 -cdrom /home/vadimr/work/isos/en_windows_server_2008_r2_standard_enterprise_datacenter_and_web_x64_dvd_x15-59754.iso -fda /home/vadimr/work/isos/virtio-drivers.vfd -cpu qemu64 -no-hpet -rtc-td-hack -net nic,vlan=0,macaddr=00:21:D8:AB:D2:A3,model=rtl8139 -net user,vlan=0 -monitor stdio -drive boot=on,index=0,if=virtio,file=/home/vadimr/work/img/w2k8r2.qcow2,cache=off -boot c
QEMU 0.9.1 monitor - type 'help' for more information

[root@localhost ~]# rpm -qa kvm
kvm-83-164.el5

[root@localhost ~]# uname -r
2.6.18-194.el5

Comment 30 James Ko 2010-04-12 18:43:02 UTC

I've done some additional investigation and testing with my build and it seems I had a misapplied patch to increase option rom size which was causing the problem.  The patch was being used to allow loading the larger gPXE option-rom with e1000 nic.

http://cvs.fedoraproject.org/viewvc/rpms/qemu/devel/qemu-roms-more-room.patch?revision=1.1&view=markup

For some reason the last change to option_rom_setup_reset() was missing.

Thank you for your help and my apologies for taking up your time.

Comment 31 Vadim Rozenfeld 2010-04-12 19:03:21 UTC

Glad to know that it works now.
Dor, can we close it?