Bug 980752
| Summary: | [virtio-win][vioser]guest bsod(9F) during reboot after install rhev-guest-tools-iso-3.1-13 | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | lijin <lijin> | ||||||
| Component: | virtio-win | Assignee: | Gal Hammer <ghammer> | ||||||
| Status: | CLOSED NOTABUG | QA Contact: | Virtualization Bugs <virt-bugs> | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | high | ||||||||
| Version: | 6.5 | CC: | acathrow, bazulay, bcao, bsarathy, byount, ghammer, iheim, jbelka, jkt, lijin, lveyde, lyarwood, michen, Rhev-m-bugs, ssekidde | ||||||
| Target Milestone: | rc | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | integration | ||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2013-08-11 11:17:01 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Bug Depends On: | |||||||||
| Bug Blocks: | 896495 | ||||||||
| Attachments: |
|
||||||||
|
Description
lijin
2013-07-03 07:40:05 UTC
1: kd> !analyze -v ******************************************************************************* * * * Bugcheck Analysis * * * ******************************************************************************* DRIVER_POWER_STATE_FAILURE (9f) A driver has failed to complete a power IRP within a specific time (usually 10 minutes). Arguments: Arg1: 00000003, A device object has been blocking an Irp for too long a time Arg2: 856a0798, Physical Device Object of the stack Arg3: 8272dae0, nt!TRIAGE_9F_POWER on Win7, otherwise the Functional Device Object of the stack Arg4: 85a8bdb8, The blocked IRP Debugging Details: ------------------ DRVPOWERSTATE_SUBCODE: 3 DRIVER_OBJECT: 85581100 IMAGE_NAME: vioser.sys DEBUG_FLR_IMAGE_TIMESTAMP: 51ac6531 MODULE_NAME: vioser FAULTING_MODULE: 8ce0f000 vioser DEFAULT_BUCKET_ID: WIN7_DRIVER_FAULT BUGCHECK_STR: 0x9F PROCESS_NAME: System CURRENT_IRQL: 2 STACK_TEXT: 8272da94 826cbf35 0000009f 00000003 856a0798 nt!KeBugCheckEx+0x1e 8272db00 826cbfae 8272dba0 00000000 8273a380 nt!PopCheckIrpWatchdog+0x1f5 8272db38 8267f039 82748a60 00000000 d956ef4a nt!PopCheckForIdleness+0x73 8272db7c 8267efdd 82730d20 8272dca8 00000001 nt!KiProcessTimerDpcTable+0x50 8272dc68 8267ee9a 82730d20 8272dca8 00000000 nt!KiProcessExpiredTimerList+0x101 8272dcdc 8267d00e 000156aa 85ba0a60 8273a380 nt!KiTimerExpiration+0x25c 8272dd20 8267ce38 00000000 0000000e 00000000 nt!KiRetireDpcList+0xcb 8272dd24 00000000 0000000e 00000000 00000000 nt!KiIdleLoop+0x38 STACK_COMMAND: kb FOLLOWUP_NAME: MachineOwner retry with qemu-ga.exe,not hit this issue. steps: 1.boot a win7-32 guest with: /usr/libexec/qemu-kvm -M rhel6.4.0 -m 2G -smp 2,sockets=2 -cpu cpu64-rhel6,+x2apic -usb -device usb-tablet -drive file=winxp.qcow2,format=qcow2,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device ide-drive,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device rtl8139,netdev=hostnet0,mac=00:52:4f:15:32:00,id=net0 -rtc-td-hack -no-kvm-pit-reinjection -chardev socket,id=111a,path=/tmp/monitor-serial,server,nowait -mon chardev=111a,mode=readline -name win7-32-serial -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -rtc base=localtime,clock=host,driftfix=slew -chardev socket,path=/tmp/tt,server,nowait,id=port22 -device isa-serial,chardev=port22,id=tt0 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/xp-vm01.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/xp-vm01.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -vnc :21 -vga cirrus -cdrom /usr/share/virtio-win/virtio-win-1.6.5.iso 2.install qemu-ga.exe in guest-agent folder of virtio-win-1.6.5.iso; 3.then intall RHEV-toolsSetup_3.1_13 4.reboot guest. guest can reboot correctly and no bsod. Two questions about the first scenario: 1. Did the RHEV-toolsSetup_3.1_13 install the balloon driver as well? 2. If you manually stop all the service that use the virtio-serial (rhev agent, spice agent, qga...) before shutting down the guest, do you get the BSOD? Thanks. (In reply to Gal Hammer from comment #5) > Two questions about the first scenario: > > 1. Did the RHEV-toolsSetup_3.1_13 install the balloon driver as well? Yes,RHEV-toolsSetup_3.1_13 installed balloon driver. > 2. If you manually stop all the service that use the virtio-serial (rhev > agent, spice agent, qga...) before shutting down the guest, do you get the > BSOD? After stop rhev-agent and spice-agent,guest did not BSOD. tetest with rhev-guest-tools-iso-3.2-11.noarch,still hit this issue. steps: 1.boot win7-32 guest /usr/libexec/qemu-kvm -M rhel6.4.0 -m 2G -smp 2,sockets=2 -cpu cpu64-rhel6,+x2apic -usb -device usb-tablet -drive file=win7-32.qcow2,format=qcow2,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device ide-drive,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device e1000,netdev=hostnet0,mac=02:52:4f:15:32:00,id=net0 -rtc-td-hack -no-kvm-pit-reinjection -chardev socket,id=111a,path=/tmp/monitor-serial,server,nowait -mon chardev=111a,mode=readline -name win7-32-serial -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=4,bus=pci.0 -chardev pty,id=channel0 -device virtserialport,chardev=channel0,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0 -vnc :22 -vga cirrus -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -rtc base=localtime,clock=host,driftfix=slew -cdrom /usr/share/rhev-guest-tools-iso/RHEV-toolsSetup_3.2_11.iso 2.install RHEV-toolsSetup_3.2_11.iso 3.reboot guest actual result: guest bsod with 9F code. expected result: guest can works normally. still,stop RHEV-agent and RHEV-spice-agent,not hit this issue. only install virtio-serial driver(in build 65) instead of installing rhev-guest-tools,not hot this issue. the attachment is the virtio-serial IRQ screenshot. Created attachment 773685 [details]
virtio-serial-IRQ-screenshoot
Hi Li, May I ask if you tested it under real RHEV env. as well ? I note that the tests were run under custom RHEL env. which has different options then what we normally have under RHEV. It's still should work fine, but it maybe the reason I wasn't able to reproduce the issue in stock RHEV. Thanks in advance, Lev Veyde. (In reply to lijin from comment #9) > tetest with rhev-guest-tools-iso-3.2-11.noarch,still hit this issue. > steps: > 1.boot win7-32 guest > /usr/libexec/qemu-kvm -M rhel6.4.0 -m 2G -smp 2,sockets=2 -cpu > cpu64-rhel6,+x2apic -usb -device usb-tablet -drive > file=win7-32.qcow2,format=qcow2,if=none,id=drive-ide0-0-0,werror=stop, > rerror=stop,cache=none -device > ide-drive,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev > tap,id=hostnet0,script=/etc/qemu-ifup -device > e1000,netdev=hostnet0,mac=02:52:4f:15:32:00,id=net0 -rtc-td-hack > -no-kvm-pit-reinjection -chardev > socket,id=111a,path=/tmp/monitor-serial,server,nowait -mon > chardev=111a,mode=readline -name win7-32-serial -device > virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=4,bus=pci.0 > -chardev pty,id=channel0 -device > virtserialport,chardev=channel0,name=com.redhat.rhevm.vdsm,bus=virtio- > serial0.0 -vnc :22 -vga cirrus -global PIIX4_PM.disable_s3=0 -global > PIIX4_PM.disable_s4=0 -rtc base=localtime,clock=host,driftfix=slew -cdrom > /usr/share/rhev-guest-tools-iso/RHEV-toolsSetup_3.2_11.iso > > 2.install RHEV-toolsSetup_3.2_11.iso > 3.reboot guest > > actual result: > guest bsod with 9F code. > > expected result: > guest can works normally. > > still,stop RHEV-agent and RHEV-spice-agent,not hit this issue. > only install virtio-serial driver(in build 65) instead of installing > rhev-guest-tools,not hot this issue. > the attachment is the virtio-serial IRQ screenshot. how does it work with IRQ instead of MSI? Thanks, Vadim. (In reply to Lev Veyde from comment #11) > Hi Li, > > May I ask if you tested it under real RHEV env. as well ? Yes .We are running tests on RHEVM3.2 env > > I note that the tests were run under custom RHEL env. which has different > options then what we normally have under RHEV. > > It's still should work fine, but it maybe the reason I wasn't able to > reproduce the issue in stock RHEV. > > Thanks in advance, > Lev Veyde. > > (In reply to Vadim Rozenfeld from comment #12) > how does it work with IRQ instead of MSI? > Thanks, > Vadim. Hi,Vadim: guest still bsod with IRQ. (In reply to Mike Cao from comment #13) > (In reply to Lev Veyde from comment #11) > > Hi Li, > > > > May I ask if you tested it under real RHEV env. as well ? > > Yes .We are running tests on RHEVM3.2 env Actually,when we run this test uder real rhevm3.2 env.,not hit this issue,guest works fine. > > > > I note that the tests were run under custom RHEL env. which has different > > options then what we normally have under RHEV. > > > > It's still should work fine, but it maybe the reason I wasn't able to > > reproduce the issue in stock RHEV. > > > > Thanks in advance, > > Lev Veyde. > > > > Created attachment 774000 [details]
screenshot-change from MSI to IRQ
(In reply to lijin from comment #16) > Created attachment 774000 [details] > screenshot-change from MSI to IRQ I see, thanks. Could you please share the crash dump file? Best regards, Vadim. (In reply to Vadim Rozenfeld from comment #17) > (In reply to lijin from comment #16) > > Created attachment 774000 [details] > > screenshot-change from MSI to IRQ > > I see, thanks. > Could you please share the crash dump file? > > Best regards, > Vadim. Hi Vadim, Please find the dump file according to #comment3. Anything thing needed,please let me know. Thanks. (In reply to lijin from comment #15) > > Yes .We are running tests on RHEVM3.2 env > > Actually,when we run this test uder real rhevm3.2 env.,not hit this > issue,guest works fine. So what's the different between these two environments? 1. What are the versions of the relevant components (qemu-kvm, drivers, tools...). 2. What are qemu's command lines? 3. Does the agents' services depends on the virtio-serial driver? (In reply to Gal Hammer from comment #23) > (In reply to lijin from comment #15) > > > > Yes .We are running tests on RHEVM3.2 env > > > > Actually,when we run this test uder real rhevm3.2 env.,not hit this > > issue,guest works fine. > > So what's the different between these two environments? > > 1. What are the versions of the relevant components (qemu-kvm, drivers, > tools...). > > 2. What are qemu's command lines? > > 3. Does the agents' services depends on the virtio-serial driver? 1.versions of the relevant components rhev environmemt: rhev-guest-tools-iso-3.2-11 qemu-kvm-rhev-0.12.1.2-2.375.el6.x86_64 kernel-2.6.32-369.el6.x86_64 our test environment: rhev-guest-tools-iso-3.2-11 qemu-kvm-rhev-0.12.1.2-2.355.el6_4.5.x86_64 kernel-2.6.32-358.6.3.el6.x86_64 2.qemu command line rhev environmemt: /usr/libexec/qemu-kvm -name win7-32-vm2 -S -M rhel6.4.0 -cpu Westmere -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -uuid 0a3cad93-2e09-45af-84e5-58d0c200d96c -smbios type=1,manufacturer=Red Hat,product=RHEV Hypervisor,version=6Server-6.4.0.4.el6,serial=4C4C4544-0054-4D10-8038-C2C04F353358,uuid=0a3cad93-2e09-45af-84e5-58d0c200d96c -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/win7-32-vm2.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2013-06-24T00:54:07,driftfix=slew -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive file=/rhev/data-center/mnt/10.66.90.115:_vol_s2bcao228623_isopool/87fa60aa-1dac-45c0-b0ca-66f8b9841d9f/images/11111111-1111-1111-1111-111111111111/en_windows_7_ultimate_x86_dvd_x15-65921.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw,serial= -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -drive file=/rhev/data-center/mnt/10.66.90.115:_vol_s2bcao228623_isopool/87fa60aa-1dac-45c0-b0ca-66f8b9841d9f/images/11111111-1111-1111-1111-111111111111/virtio-win-1.6.3_x86.vfd,if=none,id=drive-fdc0-0-0,readonly=on,format=raw,serial= -global isa-fdc.driveA=drive-fdc0-0-0 -drive file=/rhev/data-center/fab0800a-b0fb-462c-821a-07b75eac0a03/3d2facdc-1605-4ccc-8f39-e2af02c77258/images/86afae89-b1f5-4f87-8da7-e75713963098/9b84510b-de3b-4174-b3dd-2079098942c8,if=none,id=drive-virtio-disk0,format=raw,serial=86afae89-b1f5-4f87-8da7-e75713963098,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2 -netdev tap,fd=44,id=hostnet0,vhost=on,vhostfd=45 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:42:08:ef,bus=pci.0,addr=0x3,bootindex=3 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/win7-32-vm2.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/win7-32-vm2.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -spice port=5918,tls-port=5919,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -k en-us -vga qxl -global qxl-vga.ram_size=67108864 -global qxl-vga.vram_size=67108864 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 our test environment: /usr/libexec/qemu-kvm -M rhel6.4.0 -m 2G -smp core=2,threads=1,sockets=1 -cpu cpu64-rhel6,+x2apic -usb -device usb-tablet -drive file=win7-32.qcow2,format=qcow2,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device ide-drive,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device e1000,netdev=hostnet0,mac=02:52:4f:15:32:00,id=net0 -rtc-td-hack -no-kvm-pit-reinjection -chardev socket,id=111a,path=/tmp/monitor-serial,server,nowait -mon chardev=111a,mode=readline -name win7-32-serial -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=4,bus=pci.0 -chardev pty,id=channel0 -device virtserialport,chardev=channel0,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0 -vnc :22 -vga cirrus -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -rtc base=localtime,clock=host,driftfix=slew -cdrom /usr/share/rhev-guest-tools-iso/RHEV-toolsSetup_3.2_11.iso 3.Does the agents' services depends on the virtio-serial driver? Yes,I cannot install RHEV-agent without install virtio-serial driver.With service RHEV-agent started,guest bsod,if stop this servie,guest works well. (In reply to lijin from comment #24) > (In reply to Gal Hammer from comment #23) > > (In reply to lijin from comment #15) > > > > > > Yes .We are running tests on RHEVM3.2 env > > > > > > Actually,when we run this test uder real rhevm3.2 env.,not hit this > > > issue,guest works fine. > > > > So what's the different between these two environments? > > > > 1. What are the versions of the relevant components (qemu-kvm, drivers, > > tools...). > > > > 2. What are qemu's command lines? > > > > 3. Does the agents' services depends on the virtio-serial driver? > > 1.versions of the relevant components > rhev environmemt: > rhev-guest-tools-iso-3.2-11 > qemu-kvm-rhev-0.12.1.2-2.375.el6.x86_64 > kernel-2.6.32-369.el6.x86_64 > > our test environment: > rhev-guest-tools-iso-3.2-11 > qemu-kvm-rhev-0.12.1.2-2.355.el6_4.5.x86_64 > kernel-2.6.32-358.6.3.el6.x86_64 > After I update kernel and qemu-kvm-rhev and kernel to the same version of rhev environment,the problem solved,guest works well,no bsod. lijin in order to narrow down whether it is a known qemu-kvm issue .Could you help to retest it on qemu-kvm-rhev-367 & qemu-kvm-rhev-368 (In reply to Mike Cao from comment #26) > lijin > > in order to narrow down whether it is a known qemu-kvm issue .Could you help > to retest it on qemu-kvm-rhev-367 & qemu-kvm-rhev-368 retest on qemu-kvm-rhev-367 & qemu-kvm-rhev-368: with qemu-kvm-rhev-367,guest bsod; with qemu-kvm-rhev-368,guest works well,no bsod. Guys, I would also ask you to try doing the following steps: 1. add family=0xe to -cpu parameter, 2. boot the system and wait until the system updates itself, 3. restart the system. 5. perform steps 2 and 3 from #c0 Thanks, Vadim. (In reply to Vadim Rozenfeld from comment #28) > Guys, I would also ask you to try doing the following steps: > > 1. add family=0xe to -cpu parameter, > 2. boot the system and wait until the system updates itself, > 3. restart the system. > 5. perform steps 2 and 3 from #c0 > > Thanks, > Vadim. retest with qemu-kvm-rhev-367 and rhev-guest-tools-iso-3.1-12/rhev-guest-tools-iso-3.2-11,steps as comment #28,guest still bsod; (In reply to lijin from comment #29) > (In reply to Vadim Rozenfeld from comment #28) > > Guys, I would also ask you to try doing the following steps: > > > > 1. add family=0xe to -cpu parameter, > > 2. boot the system and wait until the system updates itself, > > 3. restart the system. > > 5. perform steps 2 and 3 from #c0 > > > > Thanks, > > Vadim. > > retest with qemu-kvm-rhev-367 and > rhev-guest-tools-iso-3.1-12/rhev-guest-tools-iso-3.2-11,steps as comment > #28,guest still bsod; Many thanks, Vadim. I tried with following setup and got _NO_ BSOD. w7 32bit: virtio-win 1.6.5 + 3.2 engine, vdsm-4.10.2-23.0.el6ev.x86_64, qemu-kvm-rhev-0.12.1.2-2.355.el6_4.6.x86_64 + 3.2.8 iso OK w7 64bit: virtio-win 1.6.5 + 3.2 engine, vdsm-4.10.2-23.0.el6ev.x86_64, qemu-kvm-rhev-0.12.1.2-2.355.el6_4.6.x86_64 + 3.2.8 iso OK w7 32bit: virtio-win 1.6.5 + 3.2 engine, vdsm-4.10.2-23.0.el6ev.x86_64, qemu-kvm-rhev-0.12.1.2-2.379.el6.x86_64 + 3.2.8 iso OK w7 64bit: virtio-win 1.6.5 + 3.2 engine, vdsm-4.10.2-23.0.el6ev.x86_64, qemu-kvm-rhev-0.12.1.2-2.379.el6.x86_64 + 3.2.8 iso OK (In reply to lijin from comment #27) > (In reply to Mike Cao from comment #26) > > lijin > > > > in order to narrow down whether it is a known qemu-kvm issue .Could you help > > to retest it on qemu-kvm-rhev-367 & qemu-kvm-rhev-368 > > retest on qemu-kvm-rhev-367 & qemu-kvm-rhev-368: > with qemu-kvm-rhev-367,guest bsod; > with qemu-kvm-rhev-368,guest works well,no bsod. I forgot to ask. Does both qemu-kvm-rhev packages use the same version of the seabios package? Thanks. (In reply to Gal Hammer from comment #32) > (In reply to lijin from comment #27) > > (In reply to Mike Cao from comment #26) > > > lijin > > > > > > in order to narrow down whether it is a known qemu-kvm issue .Could you help > > > to retest it on qemu-kvm-rhev-367 & qemu-kvm-rhev-368 > > > > retest on qemu-kvm-rhev-367 & qemu-kvm-rhev-368: > > with qemu-kvm-rhev-367,guest bsod; > > with qemu-kvm-rhev-368,guest works well,no bsod. > > I forgot to ask. Does both qemu-kvm-rhev packages use the same version of > the seabios package? Thanks. yes,both use the same seabios Based on comment #27, the problem is solved in qemu-kvm-rhev-368. |