Hide Forgot
Created attachment 548447 [details] Example windows balloon driver crash Description of problem: The windows balloon driver crashes very frequently when excercised repeatedly over a short period of time (10 to 20 minutes). We frequently use libvirt to move the Windows balloon driver up and down during runtime. These crashes happen even when the VM has sufficient free memory and the crashes are very reproducible. Version-Release number of selected component (if applicable): virtio-win-0.1-15.iso (From linux-kvm.org) How reproducible: It is very easily reproducible. Steps to Reproduce: 1. Create a 4GB memory Windows 7 64-bit virtual machine. Inside the VM we have a batch file that plays a 2-minute media file in an infinite loop (a cartoon video running inside the VLC media player). After the video exits, the VM plays the video over and over again. 2. While the VM is running, begin periodically invoking the "virsh setmem" command with an assortment of values, separated by a 20-second wait between invocations 3. Open virt-manager and wait for blue-screen of death to appear. Actual results: BSOD occurs after about 10 minutes. Here is an example series of "virsh setmem" commands while the VM is running. virsh setmem domain 3984588 virsh setmem domain 3786137 virsh setmem domain 3786137 virsh setmem domain 3597414 virsh setmem domain 3597414 virsh setmem domain 3418419 virsh setmem domain 3418419 virsh setmem domain 3248179 virsh setmem domain 3248179 virsh setmem domain 3086694 virsh setmem domain 3086694 virsh setmem domain 2932992 virsh setmem domain 2932992 virsh setmem domain 2787072 virsh setmem domain 2787072 virsh setmem domain 2726297 virsh setmem domain 2726297 virsh setmem domain 2863257 virsh setmem domain 3007334 virsh setmem domain 3007334 virsh setmem domain 3157862 virsh setmem domain 3157862 virsh setmem domain 3315916 virsh setmem domain 3315916 virsh setmem domain 3482572 virsh setmem domain 3482572 virsh setmem domain 3656755 virsh setmem domain 3308492 virsh setmem domain 3449548 virsh setmem domain 3449548 virsh setmem domain 3277363 virsh setmem domain 3277363 virsh setmem domain 3113932 virsh setmem domain 3113932 virsh setmem domain 2958284 virsh setmem domain 2958284 virsh setmem domain 2810419 virsh setmem domain 2810419 virsh setmem domain 2726297 virsh setmem domain 2726297 virsh setmem domain 2863257 virsh setmem domain 2863257 virsh setmem domain 3007334 virsh setmem domain 3007334 virsh setmem domain 3157862 virsh setmem domain 3157862 virsh setmem domain 3315916 virsh setmem domain 3315916 virsh setmem domain 3482572 virsh setmem domain 3482572 virsh setmem domain 3656755 virsh setmem domain 3656755 virsh setmem domain 3308492 Expected results: VM should continue running as long as there is sufficient free memory available. Additional info: At no point during this exercise did the windows "Resource Monitor" report less than 1GB of "Available Memory". I kept the resource monitor running while the VM is running using virt-manager to make sure that the balloon values were not causing any emergency low-memory situations.
Created attachment 548448 [details] Another example crash screen
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: virtio-win-0.1-15.iso qemu version: 0.15 stable
I reperformed the same test with an upgraded version of QEMU (0.15) just to rule out the possibility of the BSOD/virtio crash being due to an old qemu from RHEL 6.0, but that did not help. The crash still happens.
Can you provide the crash dump that was created during the crash? It should be in one of the following locations depending on the configuration: c:\windows\memory.dmp or in the location that will be specified in the windows that opens up after there was a crash by Windows to ask you to send the report to MS.
I've gathered several "minidump" files. The larger dumps don't seem available. Also: I'm happy to attach GDB to QEMU if you like to attempt to capture more information if that helps. I'm very familiar with debugging driver crashes in Linux. There are 6 attachments following this comment: - Three "minidumps" - And Three screen shots Please do let me know if you need me to provide you with any more sophisticated low-level information to assist in pinning down the problem.
Created attachment 548884 [details] First example crash dump
Created attachment 548885 [details] First example crash screenshot
Created attachment 548886 [details] Second example crash dump
Created attachment 548887 [details] Second example crash screenshot
Created attachment 548888 [details] Third example crash dump
Created attachment 548889 [details] Third example crash screenshot
Reproduced this issue with win2k8R2 on virtio-win-prewhql-23 steps: 1.in the guest ,play a concert video in www.youku.com 2.virsh setmem in a loop Actual Results the vedio hang at first ,then BSOD happened .
Not only BSOD happened ,sometime qemu-kvm quit with "virtio: trying to map MMIO memory" Vadim , need I report a new Bug for that ? Thanks, Mike
Full steps to reproduce this bug via qemu-kvm comamndline 1.start guest with balloon and -monitor unix:/tmp/tt,server,nowait eg:/usr/libexec/qemu-kvm -M rhel6.3.0 -enable-kvm -m 4096 -smp 2,sockets=2,cores=1,threads=1 -name win08R2 -uuid 3c9151e0-f85d-da58-c63a-8c30065eee3e -nodefconfig -nodefaults -rtc base=localtime,driftfix=slew -no-shutdown -drive file=/home/win08R2,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=/root/en_windows_server_2008_r2_standard_enterprise_datacenter_and_web_with_sp1_x64_dvd_617601.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:b7:20:34,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0 -vnc 0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -monitor unix:/tmp/tt,server,nowait 2.play a online video in the guest (I search one concert video in www.youku.com and play it ) 3.during steps2 ,run following steps . for ((;;)) do sh balloon.sh done #cat balloon.sh echo balloon `echo 3984588/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3786137/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3786137/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3597414/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3597414/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3418419/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3418419/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3248179/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3248179/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3086694/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3086694/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2932992/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2932992/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2787072/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2787072/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2726297/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2726297/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2863257/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3007334/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3007334/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3157862/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3157862/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3315916/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3315916/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3482572/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3482572/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3656755/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3308492/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3449548/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3449548/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3277363/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3277363/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3113932/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3113932/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2958284/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2958284/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2810419/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2810419/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2726297/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2726297/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2863257/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 2863257/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3007334/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3007334/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3157862/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3157862/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3315916/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3315916/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3482572/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3482572/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3656755/1024 | bc ` |nc -U /tmp/tt echo balloon `echo 3656755/1024 | bc ` |nc -U /tmp/tt Actual Results : wait less than 10mins ,sometimes guest BOSD ,sometimes qemu-kvm quit with "virtio: trying to map MMIO memory"
(In reply to comment #25) > Full steps to reproduce this bug via qemu-kvm comamndline > > 1.start guest with balloon and -monitor unix:/tmp/tt,server,nowait > 2.play a online video in the guest (I search one concert video in www.youku.com > and play it ) > 3.during steps2 ,run following steps . > for ((;;)) > do > sh balloon.sh > done > There should add a steps left step3 running 5mins ,then kill the scripts > Actual Results : > wait less than 10mins ,sometimes guest BOSD ,sometimes qemu-kvm quit with > "virtio: trying to map MMIO memory"
(In reply to comment #23) > Not only BSOD happened ,sometime qemu-kvm quit with "virtio: trying to map MMIO > memory" > > Vadim , > need I report a new Bug for that ? Hi Mike, Let's keep it as a single bug, until we find out the reason for this problem. Best regards, Vadim.
Thanks for your help, guys. Let me know if you would like me to do any testing of any kind.
We will not make it for 6.3. We would like to solve ASAP since it is a QEMU issue too (sometimes)
(In reply to comment #29) > We will not make it for 6.3. > We would like to solve ASAP since it is a QEMU issue too (sometimes) Could you please explain what makes you thinking that it is a QEMU issue? TIA, Vadim.
Hi Mike, please check the most recent balloon driver, from http://download.devel.redhat.com/brewroot/packages/virtio-win-prewhql/0.1/29/win/virtio-win-prewhql-0.1.zip Thank you, Vadim.
Reproduced it with win2k8-r2 on virtio-win-prewhql-0.1-23.Steps please refer to Comment 25 & 26. Verified it with win2k8-r2 on virtio-win-prewhql-0.1-29.Steps please also refer to Comment 25 & 26. Base on above,the bug is fixed on virtio-win-prewhql-0.1-29. Thanks Yang Zhao
According to comment #33 ,move status to VERIFIED. Vadim, Could you grant devel_ack since this bug has been fixed ald ,Thanks.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-0441.html