Bug 993614

Summary: [virtio-win][netkvm] Windows 8 32 bit crashes during HCK MPE test (BSOD D1)
Product: Red Hat Enterprise Linux 6 Reporter: Dmitry Fleytman <dfleytma>
Component: virtio-winAssignee: Dmitry Fleytman <dfleytma>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.5CC: acathrow, bcao, bsarathy, chayang, mdeng, rhod, yvugenfi
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Windows   
Whiteboard:
Fixed In Version: 67 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-01 16:43:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Dmitry Fleytman 2013-08-06 09:44:30 UTC
Description of problem:


Version-Release number of selected component (if applicable):

virtio-win-prewhql-66

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 Dmitry Fleytman 2013-08-06 11:06:14 UTC
The problem caused by race condition between DRIVER_OK flag removal from device status register, RX interrupt arrival and device queues memory cleanup.

Dump analysis:


Loading Dump File [E:\temp\Dmitry\978832_local\MEMORY.DMP]
Kernel Bitmap Dump File: Only kernel address space is available

WARNING: Path element is empty
Symbol search path is: E:\temp\RedHat\Builds\virtio-win-prewhql-66\win8\x86;;E:\temp\Dmitry\hck_symbols\x86;srv*;E:\temp\RedHat\Builds\virtio-win-prewhql-65\win8\amd64;E:\temp\Dmitry\Install\win8\x86
Executable search path is: srv*
Windows 8 Kernel Version 9200 MP (8 procs) Free x86 compatible
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 9200.16384.x86fre.win8_rtm.120725-1247
Machine Name:
Kernel base = 0x81418000 PsLoadedModuleList = 0x81602de8
Debug session time: Mon Aug  5 22:55:20.541 2013 (UTC + 3:00)
System Uptime: 0 days 4:17:11.230
Loading Kernel Symbols
...............................................................
................................................................
............
Loading User Symbols
PEB is paged out (Peb.Ldr = 7f2a400c).  Type ".hh dbgerr001" for details
Loading unloaded module list
...................
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck D1, {4, 7, 0, 8d8048c5}

Probably caused by : netkvm.sys ( netkvm!ParaNdis_VirtIODisableIrqSynchronized+29 )

Followup: MachineOwner
---------

1: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high.  This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 00000004, memory referenced
Arg2: 00000007, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write operation
Arg4: 8d8048c5, address which referenced memory

Debugging Details:
------------------


READ_ADDRESS:  00000004 

CURRENT_IRQL:  7

FAULTING_IP: 
netkvm!ParaNdis_VirtIODisableIrqSynchronized+29 [c:\cygwin\tmp\build\source\internal-kvm-guest-drivers-windows\netkvm\common\parandis-common.c @ 2496]
8d8048c5 8b4804          mov     ecx,dword ptr [eax+4]

DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

BUGCHECK_STR:  AV

PROCESS_NAME:  taskhostex.exe

TRAP_FRAME:  56f45312 -- (.trap 0x56f45312)
Unable to read trap frame at 56f45312

EXCEPTION_RECORD:  87fa32c0 -- (.exr 0xffffffff87fa32c0)
ExceptionAddress: 814edd51 (nt!KiInterruptMessageDispatch)
   ExceptionCode: 02a00016
  ExceptionFlags: 87fa32c4
NumberParameters: -2107550181
   Parameter[0]: 00000000
   Parameter[1]: 8bb65008
   Parameter[2]: 00000000
   Parameter[3]: ffffffff
   Parameter[4]: 8bb65074
   Parameter[5]: 81587400
   Parameter[6]: 00000060
   Parameter[7]: 01000705
   Parameter[8]: 00000001
   Parameter[9]: 00000001
   Parameter[10]: 00000000
   Parameter[11]: 00000001
   Parameter[12]: 00000000
   Parameter[13]: 00000000
   Parameter[14]: ffffffff

LAST_CONTROL_TRANSFER:  from 81586840 to 8150fccc

STACK_TEXT:  
9ca9f624 81586840 0000000a 00000004 00000007 nt!KiBugCheck2
9ca9f624 8d8048c5 0000000a 00000004 00000007 nt!KiTrap0E+0x2c8
9ca9f6c0 8d804063 8bb57008 00000010 8bb57008 netkvm!ParaNdis_VirtIODisableIrqSynchronized+0x29 [c:\cygwin\tmp\build\source\internal-kvm-guest-drivers-windows\netkvm\common\parandis-common.c @ 2496]
9ca9f6d8 8d80a3ce 8bb57008 9ca9f717 00000010 netkvm!ParaNdis_OnQueuedInterrupt+0x5d [c:\cygwin\tmp\build\source\internal-kvm-guest-drivers-windows\netkvm\common\parandis-common.c @ 1355]
9ca9f6f0 82615678 8bb57008 00000000 9ca9f717 netkvm!MiniportMSIInterrupt+0x1c [c:\cygwin\tmp\build\source\internal-kvm-guest-drivers-windows\netkvm\wlh\parandis6-impl.c @ 274]
9ca9f754 814edd63 87fa32c0 8bb65008 00000000 ndis!ndisMiniportMessageIsr+0x5d
9ca9f768 8158746d 87fa32c0 8bb65008 56f45312 nt!KiInterruptMessageDispatch+0x12
9ca9f768 819d352e 87fa32c0 8bb65008 56f45312 nt!KiInterruptDispatch+0x6d
9ca9f804 819b5865 803f1240 0019aa64 803ef100 hal!HalpInterruptJumpToVector+0x18e
9ca9f818 819b578d 00000001 00000202 00000007 hal!HalpInterruptLowerHardwareIrql+0xaf
9ca9f830 81449a0e 82c34000 9ca9f8a4 c04161a0 hal!KfLowerIrql+0x23
9ca9f878 81448157 00000000 00000001 00000000 nt!KeFlushMultipleRangeTb+0x137
9ca9f888 815481d4 91258d40 81548111 0000430c nt!MiFlushPteList+0x20
9ca9f964 815dfa96 82c34e28 82c34e28 8ac2ff70 nt!MmFreeSpecialPool+0x295
9ca9f9e8 818b2242 82c34e28 00000000 8ba40005 nt!ExDeferredFreePool+0xbfa
9ca9fa3c 818ab0ff 82c34e28 82c34e28 9ca9fb70 nt!VfIoFreeIrp+0x14a
9ca9fa4c 81661892 82c34e28 4889e933 816612e5 nt!IovFreeIrpPrivate+0x3c
9ca9fb70 81659b76 8ba46030 84f99d08 a10b2008 nt!IopParseDevice+0x5b1
9ca9fbf4 8165f63b 00000000 9ca9fc50 01000040 nt!ObpLookupObjectName+0x251
9ca9fc58 8165eb34 03f1d508 84f99d08 00000001 nt!ObOpenObjectByName+0xfe
9ca9fcd4 8165e110 03f1d4d0 00100080 03f1d508 nt!IopCreateFile+0x2a5
9ca9fd20 815832fc 03f1d4d0 00100080 03f1d508 nt!NtCreateFile+0x36
9ca9fd20 76f66954 03f1d4d0 00100080 03f1d508 nt!KiFastCallEntry+0x12c
WARNING: Frame IP not in any known module. Following frames may be wrong.
03f1d55c 00000000 00000000 00000000 00000000 0x76f66954


STACK_COMMAND:  kb

FOLLOWUP_IP: 
netkvm!ParaNdis_VirtIODisableIrqSynchronized+29 [c:\cygwin\tmp\build\source\internal-kvm-guest-drivers-windows\netkvm\common\parandis-common.c @ 2496]
8d8048c5 8b4804          mov     ecx,dword ptr [eax+4]

FAULTING_SOURCE_LINE:  c:\cygwin\tmp\build\source\internal-kvm-guest-drivers-windows\netkvm\common\parandis-common.c

FAULTING_SOURCE_FILE:  c:\cygwin\tmp\build\source\internal-kvm-guest-drivers-windows\netkvm\common\parandis-common.c

FAULTING_SOURCE_LINE_NUMBER:  2496

FAULTING_SOURCE_CODE:  
  2492: {
  2493:     if (interruptSource & isTransmit)
  2494:         pContext->NetSendQueue->vq_ops->disable_interrupt(pContext->NetSendQueue);
  2495:     if (interruptSource & isReceive)
> 2496:         pContext->NetReceiveQueue->vq_ops->disable_interrupt(pContext->NetReceiveQueue);
  2497:     ParaNdis_DebugHistory(pContext, hopDPC, (PVOID)0x10, interruptSource, FALSE, 0);
  2498: }
  2499: 
  2500: /**********************************************************
  2501: Common handler of PnP events


SYMBOL_STACK_INDEX:  2

SYMBOL_NAME:  netkvm!ParaNdis_VirtIODisableIrqSynchronized+29

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: netkvm

IMAGE_NAME:  netkvm.sys

DEBUG_FLR_IMAGE_TIMESTAMP:  51f76f3a

BUCKET_ID_FUNC_OFFSET:  29

FAILURE_BUCKET_ID:  AV_VRF_netkvm!ParaNdis_VirtIODisableIrqSynchronized

BUCKET_ID:  AV_VRF_netkvm!ParaNdis_VirtIODisableIrqSynchronized

Followup: MachineOwner
---------

Comment 4 Min Deng 2013-09-03 05:59:42 UTC
 Verified the bug via build 67
 build info,
    kernel-2.6.32-414.el6.x86_64   
    qemu-kvm-rhev-0.12.1.2-2.397.el6.x86_64
    virtio-win-prewhql-0.1-67
    spice-server-0.12.4-2.el6.x86_64
    seabios-0.6.1.2-28.el6.x86_64
    vgabios-0.6b-3.7.el6.noarch
 steps,
 1.boot up two guests.
 1:
 /usr/libexec/qemu-kvm \
-m 6G \
-smp 8,cores=8 \
-cpu cpu64-rhel6,+x2apic \
-usb \
-device usb-tablet \
-drive file=win8-32-nic1.raw,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none \
-device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 \
-netdev tap,sndbuf=0,id=hostnet0,vhost=on,script=/etc/qemu-ifup-private,downscript=no \
-device virtio-net-pci,netdev=hostnet0,mac=00:33:c3:02:21:21,bus=pci.0,addr=0x4,id=virtio-net-pci0 \
-netdev tap,sndbuf=0,id=hostnet2,script=/etc/qemu-ifup,downscript=no \
-device e1000,netdev=hostnet2,mac=00:22:46:c3:33:07,bus=pci.0,addr=0x6 \
-uuid 3d65a81e-4b4c-456f-a298-c7f4350b1d30 \
-no-kvm-pit-reinjection \
-chardev socket,id=111a,path=/tmp/monitor-win8-32-nic1,server,nowait \
-mon chardev=111a,mode=readline \
-vnc :1 \
-vga cirrus \
-name win8-32-67-1 \
-rtc base=localtime,clock=host,driftfix=slew \
-global PIIX4_PM.disable_s3=0 \
-global PIIX4_PM.disable_s4=0 \
-monitor stdio
-M 6.5.0
 2.submit the MPE job to HCK.

 Actual results,the MPE job on HCK passed 
 Expected results,the MPE job on HCK passed.

 Base on above,the issue has been fixed.

Comment 5 Mike Cao 2013-09-03 06:28:27 UTC
Move status to Verified according to comment #4