Bug 959815

Hi Xiangchun,

Can you upload the dump file?

Did you installed the VMs from scratch or tried to boot already installed VMs?

If you worked with already installed VM - the crash is expected (still need to check dump or at least look at the BSOD screen to confirm) because the boot device have changed. This is WinXP and Win2K3 "feature" and the only way to overcome it is to make some kind of v2v procedure that will force Windows to use the right boot device.

(In reply to comment #4)
> Did you installed the VMs from scratch or tried to boot already installed
> VMs?
Both of two scenarios hit the same problem(BSOD). Guest will show BSOD during installing. If boot already installed guest, it will BSOD as well.
> 
> If you worked with already installed VM - the crash is expected (still need
> to check dump or at least look at the BSOD screen to confirm) because the
> boot device have changed. This is WinXP and Win2K3 "feature" and the only
> way to overcome it is to make some kind of v2v procedure that will force
> Windows to use the right boot device.

I try to get guest dump file but fail when BSOD.  Now, I attached a screenshot. If dump file is needed, I'm going to do an in-depth investigation why cann't get dump file.

(In reply to comment #2)
> Hi Xiangchun,
> 
> Can you upload the dump file?

Get dump file fail. only upload a screenshot.

Created attachment 744424 [details]
BSOD screenshot

Additional question - according to the crash dump, I think that Windows 2008 and up also shouldn't work - is it a case?

(In reply to comment #8)
> Additional question - according to the crash dump, I think that Windows 2008
> and up also shouldn't work - is it a case?

no, win2008 win8 and win2012 work well. only win2003 and winxp hit this issue.

win7 64bit guest hit the same issue.

updated component
win7 works for me  with latest qemu kvm rhev.
can you please confirm?

(In reply to Michael S. Tsirkin from comment #13)
> updated component
> win7 works for me  with latest qemu kvm rhev.
> can you please confirm?

Hi Xiangchun,

Could you handle it?

Best Regards,
Junyi

Bug Verification:

Version of host:
   uname -r
   3.10.0-195.el7.x86_64
   rpm -qa|grep qemu
   qemu-kvm-rhev-2.1.2-5.el7.x86_64

steps:
1. boot guest with command:
/usr/libexec/qemu-kvm -name RHEL-Server-7.0-64 -M q35 \
-cpu SandyBridge -enable-kvm -m 4G \
-smp 1,cores=1,threads=1,sockets=1,maxcpus=160 -nodefconfig \
-spice port=5909,disable-ticketing \ 
-drive file=/home/cl/win2003-64-virtio.qcow2,if=none,id=drive-virtio-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop,media=disk,snapshot=off,bus=1,unit=1 \
-device virtio-blk-pci,scsi=on,drive=drive-virtio-disk,id=virtio-disk,bus=pcie.0,addr=0x7,bootindex=1,physical_block_size=512,logical_block_size=512 \
.......

result:
1)win7 64bit guest works well.
2)win2003 64bit guest hit the same issue.

summary:the bug hasn't fixed inside win2003 64bit guest.

Created attachment 953018 [details]
BSOD screenshot

Hi,

This is a relatively old BZ.
Can you please verify we still have this issue?

Thanks,
Marcel

(In reply to Marcel Apfelbaum from comment #23)
> Hi,
> 
> This is a relatively old BZ.
> Can you please verify we still have this issue?
> 
> Thanks,
> Marcel

I tested with the latest version until now and we still have this issue.

Versions:
Kernel: 3.10.0-325.el7.x86_64
qemu-kvm-rhev: qemu-kvm-rhev-2.3.0-31.el7.x86_64

Steps:
1.  Boot win2003 guest with 'q35', the guest BSOD.
Same command with Description.

Please try replacing this line in the command line:
-device virtio-blk-pci,scsi=on,drive=drive-virtio-disk,id=virtio-disk,bus=pcie.0,addr=0x7,bootindex=1,physical_block_size=512,logical_block_size=512 

With:
-device virtio-blk-pci,scsi=on,drive=drive-virtio-disk,id=virtio-disk,bus=pcie.0,bootindex=1,physical_block_size=512,logical_block_size=512 

With PCIe there is no meaning for the address (slot), so "addr=0x7" was 
taken out (seems to be illegal for q35)

If it works with the new command line, we should change the description to 
somethink like "qemu should not allow address in PCIe"

(In reply to Amnon Ilan from comment #26)
> Please try replacing this line in the command line:
> -device
> virtio-blk-pci,scsi=on,drive=drive-virtio-disk,id=virtio-disk,bus=pcie.0,
> addr=0x7,bootindex=1,physical_block_size=512,logical_block_size=512 
> 
> With:
> -device
> virtio-blk-pci,scsi=on,drive=drive-virtio-disk,id=virtio-disk,bus=pcie.0,
> bootindex=1,physical_block_size=512,logical_block_size=512 
> 
> With PCIe there is no meaning for the address (slot), so "addr=0x7" was 
> taken out (seems to be illegal for q35)
> 
> If it works with the new command line, we should change the description to 
> somethink like "qemu should not allow address in PCIe"


After removing "addr=0x7", qemu quit(Boot guest fail) with promoting below error messages:

'qemu-kvm: -device virtio-blk-pci,scsi=on,bus=bridge1,drive=drive-system-disk,id=system-disk,bootindex=1,ioeventfd=on: Unsupported PCI slot 0 for standard hotplug controller. Valid slots are between 1 and 31.
qemu-kvm: -device virtio-blk-pci,scsi=on,bus=bridge1,drive=drive-system-disk,id=system-disk,bootindex=1,ioeventfd=on: Device 'virtio-blk-pci' could not be initialized
'

More info:
1. Full Command with removing "addr=0x7" of virtio-blk-pci :
/usr/libexec/qemu-kvm -M q35 \
-cpu SandyBridge \
-enable-kvm -m 4G \
-smp 4,sockets=2,cores=2,threads=2 \
-no-kvm-pit-reinjection \
-name scalability-test \
-uuid 389d06a7-ed31-4fae-baf4-87bdb9b5594e \
-rtc base=localtime,clock=host,driftfix=slew \
-device pci-bridge,bus=pcie.0,id=bridge1,chassis_nr=1,addr=0x3 \
-drive file=/home/win2003-64-virtio.qcow2,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop,serial=QEMU-DISK1 \
-device virtio-blk-pci,scsi=on,bus=bridge1,drive=drive-system-disk,id=system-disk,bootindex=1,ioeventfd=on \
-netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup \
-device e1000,netdev=hostnet0,id=e1000-net-pci0,mac=08:2e:5f:0a:1d:b2,bus=bridge1,addr=0x6,bootindex=2 \
-k en-us \
-boot menu=on \
-vnc :1 \
-spice disable-ticketing,port=5931 \
-vga cirrus \
-monitor stdio

(In reply to FuXiangChun from comment #0)
> Description of problem:
> Boot win2003 or winxp with q35, they boot fail and BSOD. If use other
> machine type(pc-i440fx-1.4) guest work well. I will update dump file to bz
> asap. 
> 
>

Hi,

Regarding the "addr=0x7" thing, I didn't pay attention that the device is behind a pci-bridge, I thought it connects directly to pcie.0 bus. Since you connected it first to a bridge, addr=0x7 makes sense.

Do you use the same image (/home/win2003-64-virtio.qcow2) with both pc and q35?
Because Windows does not support switching machine types. (At least older windows like winxp and win2003)
If yes, try please to install a new img for Q35.


Thanks,
Marcel

(In reply to Marcel Apfelbaum from comment #29)
> (In reply to FuXiangChun from comment #0)
> > Description of problem:
> > Boot win2003 or winxp with q35, they boot fail and BSOD. If use other
> > machine type(pc-i440fx-1.4) guest work well. I will update dump file to bz
> > asap. 
> > 
> >
> 
> Hi,
> 
> Regarding the "addr=0x7" thing, I didn't pay attention that the device is
> behind a pci-bridge, I thought it connects directly to pcie.0 bus. Since you
> connected it first to a bridge, addr=0x7 makes sense.
> 
> Do you use the same image (/home/win2003-64-virtio.qcow2) with both pc and
> q35?
> Because Windows does not support switching machine types. (At least older
> windows like winxp and win2003)
> If yes, try please to install a new img for Q35.
> 
> 
> Thanks,
> Marcel

Hi Marcel,

No problem:)

Yes, I installed win2003 guest using pc, but booting it with q35. 

So I tried to re-install the win2003 guest using q35, the installation guest BSOD quickly(about 10s). And it has the same BSOD wrong number with Comment 7. 

More info(Full qemu command of installation):
# /usr/libexec/qemu-kvm -name win2003 \
-M q35 \
-cpu SandyBridge -m 4G,slots=256,maxmem=40G -numa node \
-smp 4,sockets=2,cores=2,threads=1 \
-uuid 82b1a01e-5f6c-4f5f-8d27-3855a74e6b6c \
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16 \
-spice port=5902,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-migration=on \
-drive file=/home/win2003-64-virtio.qcow2,format=qcow2,if=none,id=drive-virtio-blk0,werror=stop,rerror=stop \
-device virtio-blk-pci,drive=drive-virtio-blk0,id=virtio-blk0,indirect_desc=off \
-netdev tap,id=hostnet1 \
-device virtio-net-pci,netdev=hostnet1,id=net1,mac=12:54:00:5c:88:62 \
-qmp tcp:0:5555,server,nowait \
-monitor stdio \
-drive file=/home/en_win_srv_2003_r2_enterprise_x64_with_sp2_cd1_X13-06188.iso,if=none,id=drive-ide0-0-1,readonly=on,format=raw \
-device ide-cd,drive=drive-ide0-0-1,bootindex=0,id=ide0-0-1 \
-fda  /usr/share/virtio-win/virtio-win-1.8.0_amd64.vfd \


-Pei

(In reply to Pei Zhang from comment #30)
> (In reply to Marcel Apfelbaum from comment #29)
> > (In reply to FuXiangChun from comment #0)
> > > Description of problem:
> > > Boot win2003 or winxp with q35, they boot fail and BSOD. If use other
> > > machine type(pc-i440fx-1.4) guest work well. I will update dump file to bz
> > > asap. 
> > > 
> > >
> > 
> > Hi,
> > 
> > Regarding the "addr=0x7" thing, I didn't pay attention that the device is
> > behind a pci-bridge, I thought it connects directly to pcie.0 bus. Since you
> > connected it first to a bridge, addr=0x7 makes sense.
> > 
> > Do you use the same image (/home/win2003-64-virtio.qcow2) with both pc and
> > q35?
> > Because Windows does not support switching machine types. (At least older
> > windows like winxp and win2003)
> > If yes, try please to install a new img for Q35.
> > 
> > 
> > Thanks,
> > Marcel
> 
> Hi Marcel,
> 
> No problem:)
> 
> Yes, I installed win2003 guest using pc, but booting it with q35. 
> 
> So I tried to re-install the win2003 guest using q35, the installation guest
> BSOD quickly(about 10s). And it has the same BSOD wrong number with Comment
> 7. 

Thanks for your patience! :)

The image file is clean, right? Is a clean
install on a an empty disk? And windows 2003 selects the right virtio drivers from the CD you provided? The virtio drivers are the signed ones?

If all the above is right, I'll start working to reproduce this.
Thanks,
Marcel

> 
> More info(Full qemu command of installation):
> # /usr/libexec/qemu-kvm -name win2003 \
> -M q35 \
> -cpu SandyBridge -m 4G,slots=256,maxmem=40G -numa node \
> -smp 4,sockets=2,cores=2,threads=1 \
> -uuid 82b1a01e-5f6c-4f5f-8d27-3855a74e6b6c \
> -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16 \
> -spice
> port=5902,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-
> migration=on \
> -drive
> file=/home/win2003-64-virtio.qcow2,format=qcow2,if=none,id=drive-virtio-blk0,
> werror=stop,rerror=stop \
> -device
> virtio-blk-pci,drive=drive-virtio-blk0,id=virtio-blk0,indirect_desc=off \
> -netdev tap,id=hostnet1 \
> -device virtio-net-pci,netdev=hostnet1,id=net1,mac=12:54:00:5c:88:62 \
> -qmp tcp:0:5555,server,nowait \
> -monitor stdio \
> -drive
> file=/home/en_win_srv_2003_r2_enterprise_x64_with_sp2_cd1_X13-06188.iso,
> if=none,id=drive-ide0-0-1,readonly=on,format=raw \
> -device ide-cd,drive=drive-ide0-0-1,bootindex=0,id=ide0-0-1 \
> -fda  /usr/share/virtio-win/virtio-win-1.8.0_amd64.vfd \
> 
> 
> -Pei

The "need info" is for comment 31.

Thanks,
Marcel

(In reply to Marcel Apfelbaum from comment #31)
... 
> 
> Thanks for your patience! :)
> 
> The image file is clean, right? Is a clean install on a an empty disk? 

Yes, I deleted the old one and create a new one every time. So I used a new empty qcow2 image before installing, it's clean.

> And windows 2003 selects the right virtio
> drivers from the CD you provided? The virtio drivers are the signed ones?

versions: virtio-win-1.8.0-4.el7.noarch
I used the above signed one, not prewhql.

> If all the above is right, I'll start working to reproduce this.
> Thanks,
> Marcel
> 
...

More Info:
1. I re-install win2003 using same command in Comment 30, but replacing 'q35' with 'pc', the guest installation works well. 

2. I tried win7, 'pc' and 'q35' both work well for installing.

Hi,

There are two problems:
1. Q35 AHCI driver is not part of Windows disk for Win2013 and possibly WinXP
    Solution: use legacy ide controller:
     -device piix3-ide,id=legacyide \
     -drive file=winxp-q35.qcow2,if=none,id=disk \
     -device ide-hd,drive=disk,bus=legacyide.0  \
     -drive file=xp_sp3.iso,if=none,id=cdrom    \
     -device ide-cd,drive=cdrom,bus=legacyide.

2. An AHCI bug preventing Windows to start. A patch was submitted upstream:
  - https://www.mail-archive.com/qemu-devel@nongnu.org/msg357884.html

It can already be checked that the problem is solved using upstream QEMU with the mentioned patch and the above command line.

(In reply to Amnon Ilan from comment #26)
> Please try replacing this line in the command line:
> -device
> virtio-blk-pci,scsi=on,drive=drive-virtio-disk,id=virtio-disk,bus=pcie.0,
> addr=0x7,bootindex=1,physical_block_size=512,logical_block_size=512 
> 
> With:
> -device
> virtio-blk-pci,scsi=on,drive=drive-virtio-disk,id=virtio-disk,bus=pcie.0,
> bootindex=1,physical_block_size=512,logical_block_size=512 
> 
> With PCIe there is no meaning for the address (slot), so "addr=0x7" was 
> taken out (seems to be illegal for q35)

This is not correct for the pcie root complex - it allows addr up to 0x1f. Different PCIE controllers have different ranges of acceptable addrs. In particular, pcie.0 (called pcie-root by libvirt) and x3130-upstream (called pcie-switch-upstream-port by libvirt) allow addr between 0 and 0x1f, while ioh3420 and xio3130-downstream (pcie-root-port and pcie-switch-downstream-port) have only a single port that is at address 0.

On to Marcel's finding - I know that later in the life of WinXP (SP2 or something? I'm no connoisseur of Windows flavors)\, they provided an AHCI driver disk, and many corporate-types modified their Windows installing (I think they called it "slip streaming"??) to have the AHCI driver included in their initial install media.

libvirt has avoided adding support for "extra" IDE controllers up until now because we didn't want to encourage more use of an outdated and very slow method of connecting disks. We could add such support, but then of course would have to support it essentially forever, so we'd prefer to avoid it.

Here, though, is the full list of possible ways to overcome this problem:

1) add support for explicitly configured IDE controllers so that a Q35 machine could have such a controller added, and disks attached to that controller rather than the achi controller if the guest is WinXP (and also Win2003) (and Win7?? Really???).

2) have someone knowledgeable about Windows look into the "slipstream an AHCI controller" method of setting up installable WinXP media and document this as the method of installing WinXP when the machinetype is Q35.

3) Document that for WinXP (and any other ancient OSes that require IDE) the proper method of running them is to switch machinetype to Q35.

If WinXP won't be officially supported, then I think that either of (2) or (3) would be acceptable.

Another possible solution that was brought up in an IRC discussion:

4) add support in the BIOS (?) to switch the sata controller between AHCI and IDE mode.

Created libvirt clone : https://bugzilla.redhat.com/show_bug.cgi?id=1316619