Bug 989459 - Windows guest stuck at bootng(using scsi-hd)
Windows guest stuck at bootng(using scsi-hd)
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm (Show other bugs)
7.0
Unspecified Unspecified
urgent Severity urgent
: rc
: ---
Assigned To: Vadim Rozenfeld
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-29 06:07 EDT by Xiaoqing Wei
Modified: 2014-09-09 03:48 EDT (History)
11 users (show)

See Also:
Fixed In Version: virtio-win-prewhql-88
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-09-09 03:48:37 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
pic (171.56 KB, image/png)
2013-09-11 07:22 EDT, Xiaoqing Wei
no flags Details

  None (edit)
Description Xiaoqing Wei 2013-07-29 06:07:22 EDT
Description of problem:

Win2012r2 stuck at boot during os installation(using scsi-hd and aio=threads)

Version-Release number of selected component (if applicable):

kernel-3.10.0-2.el7.x86_64
qemu-kvm-1.5.2-1.el7.x86_64
virtio-win-prewhql-65

How reproducible:
only once, tried another 4 rounds, didn't manage to reproduce.

Steps to Reproduce:
1.
/root/staf-kvm-devel/autotest-devel/client/tests/virt/qemu/qemu \
    -S \
    -name 'virt-tests-vm1' \
    -nodefaults \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20130726-143052-9Z2qISeN,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control \
    -chardev socket,id=serial_id_serial1,path=/tmp/serial-serial1-20130726-143052-9Z2qISeN,server,nowait \
    -device isa-serial,chardev=serial_id_serial1 \
    -chardev socket,id=seabioslog_id_20130726-143052-9Z2qISeN,path=/tmp/seabios-20130726-143052-9Z2qISeN,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20130726-143052-9Z2qISeN,iobase=0x402 \
    -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=0x4 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,addr=0x5 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/images/win2012r2-64.raw',if=none,id=virtio-scsi-id0,media=disk,cache=none,snapshot=off,format=raw,aio=threads \
    -device scsi-hd,drive=virtio-scsi-id0 \
    -device e1000,netdev=idODbbMw,mac='9a:0a:0b:0c:0d:0e',bus=pci.0,addr=0x3,id='idcKhjhx' \
    -netdev tap,id=idODbbMw,fd=22 \
    -m 4096 \
    -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \
    -cpu 'SandyBridge' \
    -M pc \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/ISO/Win2012R2/en_windows_server_2012_r2_datacenter_preview_x64_dvd_2358570.iso',if=none,id=virtio-scsi-id1,media=cdrom,readonly=on,format=raw \
    -device scsi-cd,drive=virtio-scsi-id1 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso',if=none,id=virtio-scsi-id2,media=cdrom,readonly=on,format=raw \
    -device scsi-cd,drive=virtio-scsi-id2 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/virtio-win.iso',if=none,id=virtio-scsi-id3,media=cdrom,readonly=on,format=raw \
    -device scsi-cd,drive=virtio-scsi-id3 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/images/win2012r2-64/answer.vfd',if=none,id=fdc0-0-0,media=disk,readonly=off,format=raw \
    -global isa-fdc.driveA=fdc0-0-0 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
    -spice password=123456,addr=0,tls-port=3200,x509-dir=/tmp/spice_x509d,tls-channel=main,tls-channel=inputs,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4 \
    -vga qxl \
    -global qxl-vga.vram_size=33554432 \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=d,menu=off  \
    -enable-kvm
2.
3.

Actual results:
Win2012r2 stuck at boot during os installation
last for at least 30mins before vm killed.
Expected results:
installation finished successfully, not stuck
Additional info:
same test work well when using aio=native




processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 42
model name	: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
stepping	: 7
microcode	: 0x28
cpu MHz		: 1904.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 6784.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:
Comment 3 Xiaoqing Wei 2013-07-29 08:18:34 EDT
BTW, the guest os is not died when the stuck happened,

press the combination key: 'shift+F10' can open a cmd in guest.
Comment 4 Xiaoqing Wei 2013-08-06 22:39:11 EDT
met it again with 
qemu-kvm-1.5.2-2.el7.x86_64
3.10.0-3.el7.x86_64
Comment 5 Xiaoqing Wei 2013-08-07 04:54:12 EDT
This could also be triggered when using ide-cdrom(hard drive still scsi-hd)

/root/staf-kvm-devel/autotest-devel/client/tests/virt/qemu/qemu \
    -S \
    -name 'virt-tests-vm1' \
    -nodefaults \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20130801-160234-9EJouOL8,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control \
    -chardev socket,id=serial_id_serial1,path=/tmp/serial-serial1-20130801-160234-9EJouOL8,server,nowait \
    -device isa-serial,chardev=serial_id_serial1 \
    -chardev socket,id=seabioslog_id_20130801-160234-9EJouOL8,path=/tmp/seabios-20130801-160234-9EJouOL8,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20130801-160234-9EJouOL8,iobase=0x402 \
    -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=0x4 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,addr=0x5 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/images/win2012r2-64.qcow2',if=none,id=virtio-scsi0-id0,media=disk,cache=none,snapshot=off,format=qcow2,aio=threads \
    -device scsi-hd,drive=virtio-scsi0-id0 \
    -device e1000,netdev=idb6MnUd,mac='9a:bb:bc:bd:be:bf',bus=pci.0,addr=0x3,id='idrud9iv' \
    -netdev tap,id=idb6MnUd,fd=22 \
    -m 4096 \
    -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \
    -cpu 'SandyBridge' \
    -M pc \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/ISO/Win2012R2/en_windows_server_2012_r2_datacenter_preview_x64_dvd_2358570.iso',index=1,if=none,id=drive-ide0-0-0,media=cdrom,format=raw \
    -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso',index=2,if=none,id=drive-ide0-0-1,media=cdrom,format=raw \
    -device ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/virtio-win.iso',index=3,if=none,id=drive-ide0-1-0,media=cdrom,format=raw \
    -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/images/win2012r2-64/answer.vfd',if=none,id=fdc0-0-0,media=disk,readonly=off,format=raw \
    -global isa-fdc.driveA=fdc0-0-0 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
    -vnc :0 \
    -vga std \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=d,menu=off  \
    -enable-kvm
Comment 6 Suqin Huang 2013-08-27 06:09:10 EDT
Hi xwei,
it only happen once, why do you think it's aio ans scsi-hd related.

if it's aio related can you provide the result with the following steps

I'm afraid the aio threads are not created by main thread.

1. boot guest with 4 vcpus.

2. stop guest

3. pin main thread to host cpu 1

4. pin one vcpu thread to one host cpu.

5. cont guest

get output with the following cmd:


import commands

thread_cmd = "ps -C qemu-kvm H -o spid="
thread_output = commands.getoutput(thread_cmd).split()

for i in range(len(thread_output)):                                             
    taskset_cmd = "taskset -p %s" % thread_output[i]
    taskset_cpu = commands.getoutput(taskset_cmd)
    print taskset_cpu
Comment 8 ShupingCui 2013-09-11 03:16:28 EDT
met it again with:
kernel-2.6.32-416.el6.x86_64
qemu-kvm-0.12.1.2-2.400.el6.x86_64
virtio-win-prewhql-0.1-69

stuck with virtio-scsi, and installation finished with virtio-blk.
Comment 9 ShupingCui 2013-09-11 06:10:20 EDT
command line:

/usr/libexec/qemu-kvm \
    -name 'virt-tests-vm1' \
    -nodefaults \
    -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=0x4 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,addr=0x5 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/images/win2012-64-virtio.raw',if=none,id=virtio-scsi0-id0,media=disk,cache=none,snapshot=off,format=raw,aio=threads \
    -device scsi-hd,drive=virtio-scsi0-id0 \
    -device virtio-net-pci,netdev=idvNJoj4,mac='9a:8e:8f:90:91:92',bus=pci.0,addr=0x3,id='idpuEowM' \
    -netdev tap,id=idvNJoj4,vhost=on,vhostfd=20,fd=18 \
    -m 8192 \
    -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \
    -cpu 'SandyBridge' \
    -M rhel6.5.0 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/ISO/Win2012/en_windows_server_2012_x64_dvd_915478.iso',if=none,id=virtio-scsi1-id1,media=cdrom,readonly=on,format=raw \
    -device scsi-cd,drive=virtio-scsi1-id1 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso',if=none,id=virtio-scsi2-id2,media=cdrom,readonly=on,format=raw \
    -device scsi-cd,drive=virtio-scsi2-id2 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/virtio-win-prewhql-0.1-69.iso',if=none,id=virtio-scsi3-id3,media=cdrom,readonly=on,format=raw \
    -device scsi-cd,drive=virtio-scsi3-id3 \
    -device AC97,addr=0x6 \
    -drive file='/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/images/win2012-64/answer.vfd',if=none,id=fdc0-0-0,media=disk,readonly=off,format=raw \
    -global isa-fdc.driveA=fdc0-0-0 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
    -spice port=3000,password=123456,addr=0,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4 \
    -vga qxl \
    -global qxl-vga.vram_size=33554432 \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=d,menu=off  \
    -enable-kvm

Host cpuinfo:
processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 58
model name	: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
stepping	: 9
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
bogomips	: 6784.78
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:
Comment 10 Xiaoqing Wei 2013-09-11 07:22:40 EDT
Created attachment 796346 [details]
pic

this is happen when guest loading the virtio-scsi driver,
the progress bar is keep moving from left to right, and reset to left.
I observed this for at least 5 min

and on -69 prewhql, it can happend whatever native/threads.
now 100% on -69

and can happen on win2012 / win7 too, raise to urgent/urgent
Comment 11 Xiaoqing Wei 2013-12-17 06:59:49 EST
Found that this could also happen on OS booting, 2012 R2 Server(with gui)
Comment 12 Ronen Hod 2014-01-02 05:28:34 EST
QE,

Please retest with the latest virtio-win. There were relevant bug fixes in build 74.
Thanks.
Can you attach the unattended install (unattended.ini or something)
Comment 13 juzhang 2014-01-02 20:36:58 EST
(In reply to Ronen Hod from comment #12)
> QE,
> 
> Please retest with the latest virtio-win. There were relevant bug fixes in
> build 74.
> Thanks.
> Can you attach the unattended install (unattended.ini or something)

Hi Xwei,

Could you have a try and update the result in bz?

Best Regards,
Junyi
Comment 14 Xiaoqing Wei 2014-01-02 21:43:13 EST
Oh, Hello Ronen, Junyi,

Sorry for late response, the C#11 was using -74 virtio-win-prewhql
We shifted to it Since Nov 21, 2013 CST.

https://brewweb.devel.redhat.com/packageinfo?packageID=31338
Comment 17 Vadim Rozenfeld 2014-07-04 10:12:03 EDT
Any updates on this issue?
Is it still reproducible on rhel7.1 and the latest drivers?

Thanks,
Vadim.
Comment 18 Ronen Hod 2014-08-06 05:47:51 EDT
QE,
Since it doesn't reproduce on Vadim's setup,
Please test again with build-88
If it reproduces, Vadim will provide a debug version of the drivers in order to get a log through the serial port.
Comment 19 juzhang 2014-08-11 04:56:02 EDT
(In reply to Ronen Hod from comment #18)
> QE,
> Since it doesn't reproduce on Vadim's setup,
> Please test again with build-88
> If it reproduces, Vadim will provide a debug version of the drivers in order
> to get a log through the serial port.

Hi Scui,

Could you give a help for handling this?

Best Regards,
Junyi
Comment 20 ShupingCui 2014-09-03 03:21:26 EDT
Re-test 10 times with virtio-win-prewhql-88, cannot reproduce this bug.
Comment 21 Vadim Rozenfeld 2014-09-03 06:07:24 EDT
Thanks,
Can we close it?
Comment 23 ShupingCui 2014-09-08 09:58:42 EDT
(In reply to Vadim Rozenfeld from comment #21)
> Thanks,
> Can we close it?

Hi Vadim,

I think we can close this bug.

Thanks,
Shuping

Note You need to log in before you can comment on or make changes to this bug.