Bug 1313243 - [virtio-win][netkvm][rhel6]win2008-32 guest bsod with HARDWARE_INTERRUP_STORM(f2) when running netperf for longevity test
[virtio-win][netkvm][rhel6]win2008-32 guest bsod with HARDWARE_INTERRUP_STORM...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: virtio-win (Show other bugs)
7.3
x86_64 Windows
unspecified Severity high
: rc
: ---
Assigned To: Yan Vugenfirer
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-03-01 03:54 EST by wehao
Modified: 2016-11-04 04:52 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
NO_DOCS
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-11-04 04:52:05 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description wehao 2016-03-01 03:54:04 EST
Description of problem:
win2008-32 guest bsod with HARDWARE_INTERRUP_STORM(f2) when running netperf for longevity test

Version-Release number of selected component (if applicable):
kernel-debug-2.6.32-620.el6.x86_64.rpm
qemu-kvm-rhev-0.12.1.2-2.489.el6.x86_64
seabios-0.6.1.2-30.el6.x86_64
virtio-win-1.7.5-0.el6.noarch.rpm

How reproducible:
100%
The first BSOD occurred several dozen hours after starting netperf.exe and then BSOD occurs nearly as soon as netperf.exe starting.

Steps to Reproduce:
1.boot guest with:
/usr/libexec/qemu-kvm \
-name win2008-32 \
-enable-kvm -m 4G -smp 4 -uuid 6888677e-d765-44d0-986c-daa923b610f8 \
-nodefconfig --nodefaults -boot order=cd,menu=on -drive file=win.raw,if=none,id=drive-ide0-0-0,format=raw,serial=mike_cao,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=en_windows_server_2008_datacenter_enterprise_standard_sp2_checked_build_x86_dvd_342434.iso,media=cdrom,id=cdrom,if=none -device ide-drive,drive=cdrom,bootindex=1 -cdrom /usr/share/virtio-win/virtio-win.iso -vnc 0.0.0.0:7 -vga cirrus -monitor stdio -qmp tcp:0:4445,server,nowait -netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:22:32:37:88:71

2.run netperf several hours with:
cmd >for /l %k in (1,1,10000) do for %j in (TCP_STREAM,UDP_STREAM) do for %i in (32 64 128 256 512 1024 2048 4096 8192 16384 32768) do NETPERF -H netserver's ip -c -C -t %j -f m -l 180s -- -m %i


Actual results:
bsod with HARDWARE_INTERRUP_STORM(f2)

Expected results:
normal

Additional info:
0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

HARDWARE_INTERRUPT_STORM (f2)
Note:  This bug check no longer exists in any version of Windows.
This bugcheck will show up on the screen when the kernel
detects an interrupt "storm".  An interrupt storm is defined as a level
triggered interrupt signal staying in the asserted state.  This is fatal
to the system in the manner that the system will hard hang, or "bus lock".
This can happen because of the following:
 -  A piece of hardware does not release its interrupt signal after being told
    to do so by the device driver
 -  A device driver does not instruct its hardware to release the interrupt
    signal because it does not believe the interrupt was initiated from its
    hardware
 -  A device driver claims the interrupt even though the interrupt was not
    initiated from its hardware.  Note that this can only occur when multiple
    devices are sharing the same IRQ.
 -  The ELCR (edge level control register) is set incorrectly.
 -  Edge and Level interrupt triggered devices share an IRQ (e.g. COM port and
    PCI SCSI controller).
All of these cases will instantly hard hang your system.  Instead of hard
hanging the system, this bugcheck is initiated since in many cases it can
identify the culprit.
When the bugcheck occurs, the module containing the ISR (interrupt service
routine) of the storming IRQ is displayed on the screen.  This is an
example of what you would see:
*** STOP: 0x000000F2 (0xFCA7C55C, 0x817B9B28, 0x817D2AA0, 0x00000002)
An interrupt storm has caused the system to hang.
*** Address FCA7C55C base at FCA72000, Datestamp 3A72BDEF - ACPI.sys
In the event the fourth parameter is a 0x00000001, the module pointed to
is very likely the culprit.  Either the driver is broken, or the hardware
is malfunctioning.
In the event the fourth parameter is a 0x00000002, the module pointed to
is the first ISR in the chain, and is never guaranteed to be the culprit.
A user experiencing this bugcheck repeatedly should try to isolate the
problem by looking for devices that are on the same IRQ as the one for
which the module is a driver for (in this case, the same IRQ that ACPI
is using).  In the future, we may be able to list all devices on a
chained ISR.
Arguments:
Arg1: 81460e58, address of the ISR (or first ISR in the chain) connected to the storming interrupt vector
Arg2: 86114000, ISR context value
Arg3: 8611bc80, address of the interrupt object for the storming interrupt vector
Arg4: 00000000, 0x1 if the ISR is not chained, 0x2 if the ISR is chained

Debugging Details:
------------------

***** Kernel symbols are WRONG. Please fix symbols to do analysis.

*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!KPRCB                                      ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!KPRCB                                      ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************

ADDITIONAL_DEBUG_TEXT:  
You can run '.symfix; .reload' to try to fix the symbol path and load symbols.

MODULE_NAME: netkvm

FAULTING_MODULE: 8141e000 nt

DEBUG_FLR_IMAGE_TIMESTAMP:  556edb83

DEFAULT_BUCKET_ID:  VISTA_DRIVER_FAULT

BUGCHECK_STR:  0xF2

CURRENT_IRQL:  0

ANALYSIS_VERSION: 6.3.9600.16384 (debuggers(dbg).130821-1623) amd64fre

LAST_CONTROL_TRANSFER:  from 81591cb7 to 81459ba2

STACK_TEXT:  
WARNING: Stack unwind information not available. Following frames may be wrong.
815b4c1c 81591cb7 000000f2 81460e58 86114000 nt!KeBugCheckEx+0x1e
815b4c78 8d3b48da 00000000 00000000 86111958 nt!KeSynchronizeExecution+0x5f7
815b4ce4 81caaeaa 864fc0e8 815b9a90 815b4d50 netkvm+0x28da
815b4cf4 8154c449 00000000 00000000 815bb940 hal!HalAcquireDisplayOwnership+0x91e
815b4d50 815925af 00000000 0000000e 00000000 nt!PoSetPowerState+0x95d
815b4d54 00000000 0000000e 00000000 00000000 nt!KiDispatchInterrupt+0x63f


STACK_COMMAND:  kb

FOLLOWUP_IP: 
netkvm+28da
8d3b48da 84c0            test    al,al

SYMBOL_STACK_INDEX:  2

SYMBOL_NAME:  netkvm+28da

FOLLOWUP_NAME:  MachineOwner

IMAGE_NAME:  netkvm.sys

BUCKET_ID:  WRONG_SYMBOLS

FAILURE_BUCKET_ID:  WRONG_SYMBOLS

ANALYSIS_SOURCE:  KM

FAILURE_ID_HASH_STRING:  km:wrong_symbols

FAILURE_ID_HASH:  {70b057e8-2462-896f-28e7-ac72d4d365f8}

Followup: MachineOwner
---------

0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

HARDWARE_INTERRUPT_STORM (f2)
Note:  This bug check no longer exists in any version of Windows.
This bugcheck will show up on the screen when the kernel
detects an interrupt "storm".  An interrupt storm is defined as a level
triggered interrupt signal staying in the asserted state.  This is fatal
to the system in the manner that the system will hard hang, or "bus lock".
This can happen because of the following:
 -  A piece of hardware does not release its interrupt signal after being told
    to do so by the device driver
 -  A device driver does not instruct its hardware to release the interrupt
    signal because it does not believe the interrupt was initiated from its
    hardware
 -  A device driver claims the interrupt even though the interrupt was not
    initiated from its hardware.  Note that this can only occur when multiple
    devices are sharing the same IRQ.
 -  The ELCR (edge level control register) is set incorrectly.
 -  Edge and Level interrupt triggered devices share an IRQ (e.g. COM port and
    PCI SCSI controller).
All of these cases will instantly hard hang your system.  Instead of hard
hanging the system, this bugcheck is initiated since in many cases it can
identify the culprit.
When the bugcheck occurs, the module containing the ISR (interrupt service
routine) of the storming IRQ is displayed on the screen.  This is an
example of what you would see:
*** STOP: 0x000000F2 (0xFCA7C55C, 0x817B9B28, 0x817D2AA0, 0x00000002)
An interrupt storm has caused the system to hang.
*** Address FCA7C55C base at FCA72000, Datestamp 3A72BDEF - ACPI.sys
In the event the fourth parameter is a 0x00000001, the module pointed to
is very likely the culprit.  Either the driver is broken, or the hardware
is malfunctioning.
In the event the fourth parameter is a 0x00000002, the module pointed to
is the first ISR in the chain, and is never guaranteed to be the culprit.
A user experiencing this bugcheck repeatedly should try to isolate the
problem by looking for devices that are on the same IRQ as the one for
which the module is a driver for (in this case, the same IRQ that ACPI
is using).  In the future, we may be able to list all devices on a
chained ISR.
Arguments:
Arg1: 81460e58, address of the ISR (or first ISR in the chain) connected to the storming interrupt vector
Arg2: 86114000, ISR context value
Arg3: 8611bc80, address of the interrupt object for the storming interrupt vector
Arg4: 00000000, 0x1 if the ISR is not chained, 0x2 if the ISR is chained

Debugging Details:
------------------

***** Kernel symbols are WRONG. Please fix symbols to do analysis.

*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!KPRCB                                      ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!KPRCB                                      ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************

ADDITIONAL_DEBUG_TEXT:  
You can run '.symfix; .reload' to try to fix the symbol path and load symbols.

MODULE_NAME: netkvm

FAULTING_MODULE: 8141e000 nt

DEBUG_FLR_IMAGE_TIMESTAMP:  556edb83

DEFAULT_BUCKET_ID:  VISTA_DRIVER_FAULT

BUGCHECK_STR:  0xF2

CURRENT_IRQL:  0

ANALYSIS_VERSION: 6.3.9600.16384 (debuggers(dbg).130821-1623) amd64fre

LAST_CONTROL_TRANSFER:  from 81591cb7 to 81459ba2

STACK_TEXT:  
WARNING: Stack unwind information not available. Following frames may be wrong.
815b4c1c 81591cb7 000000f2 81460e58 86114000 nt!KeBugCheckEx+0x1e
815b4c78 8d3b48da 00000000 00000000 86111958 nt!KeSynchronizeExecution+0x5f7
815b4ce4 81caaeaa 864fc0e8 815b9a90 815b4d50 netkvm+0x28da
815b4cf4 8154c449 00000000 00000000 815bb940 hal!HalAcquireDisplayOwnership+0x91e
815b4d50 815925af 00000000 0000000e 00000000 nt!PoSetPowerState+0x95d
815b4d54 00000000 0000000e 00000000 00000000 nt!KiDispatchInterrupt+0x63f


STACK_COMMAND:  kb

FOLLOWUP_IP: 
netkvm+28da
8d3b48da 84c0            test    al,al

SYMBOL_STACK_INDEX:  2

SYMBOL_NAME:  netkvm+28da

FOLLOWUP_NAME:  MachineOwner

IMAGE_NAME:  netkvm.sys

BUCKET_ID:  WRONG_SYMBOLS

FAILURE_BUCKET_ID:  WRONG_SYMBOLS

ANALYSIS_SOURCE:  KM

FAILURE_ID_HASH_STRING:  km:wrong_symbols

FAILURE_ID_HASH:  {70b057e8-2462-896f-28e7-ac72d4d365f8}

Followup: MachineOwner
---------

0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

HARDWARE_INTERRUPT_STORM (f2)
Note:  This bug check no longer exists in any version of Windows.
This bugcheck will show up on the screen when the kernel
detects an interrupt "storm".  An interrupt storm is defined as a level
triggered interrupt signal staying in the asserted state.  This is fatal
to the system in the manner that the system will hard hang, or "bus lock".
This can happen because of the following:
 -  A piece of hardware does not release its interrupt signal after being told
    to do so by the device driver
 -  A device driver does not instruct its hardware to release the interrupt
    signal because it does not believe the interrupt was initiated from its
    hardware
 -  A device driver claims the interrupt even though the interrupt was not
    initiated from its hardware.  Note that this can only occur when multiple
    devices are sharing the same IRQ.
 -  The ELCR (edge level control register) is set incorrectly.
 -  Edge and Level interrupt triggered devices share an IRQ (e.g. COM port and
    PCI SCSI controller).
All of these cases will instantly hard hang your system.  Instead of hard
hanging the system, this bugcheck is initiated since in many cases it can
identify the culprit.
When the bugcheck occurs, the module containing the ISR (interrupt service
routine) of the storming IRQ is displayed on the screen.  This is an
example of what you would see:
*** STOP: 0x000000F2 (0xFCA7C55C, 0x817B9B28, 0x817D2AA0, 0x00000002)
An interrupt storm has caused the system to hang.
*** Address FCA7C55C base at FCA72000, Datestamp 3A72BDEF - ACPI.sys
In the event the fourth parameter is a 0x00000001, the module pointed to
is very likely the culprit.  Either the driver is broken, or the hardware
is malfunctioning.
In the event the fourth parameter is a 0x00000002, the module pointed to
is the first ISR in the chain, and is never guaranteed to be the culprit.
A user experiencing this bugcheck repeatedly should try to isolate the
problem by looking for devices that are on the same IRQ as the one for
which the module is a driver for (in this case, the same IRQ that ACPI
is using).  In the future, we may be able to list all devices on a
chained ISR.
Arguments:
Arg1: 81460e58, address of the ISR (or first ISR in the chain) connected to the storming interrupt vector
Arg2: 86114000, ISR context value
Arg3: 8611bc80, address of the interrupt object for the storming interrupt vector
Arg4: 00000000, 0x1 if the ISR is not chained, 0x2 if the ISR is chained

Debugging Details:
------------------

***** Kernel symbols are WRONG. Please fix symbols to do analysis.

*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!KPRCB                                      ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!KPRCB                                      ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************

ADDITIONAL_DEBUG_TEXT:  
You can run '.symfix; .reload' to try to fix the symbol path and load symbols.

MODULE_NAME: netkvm

FAULTING_MODULE: 8141e000 nt

DEBUG_FLR_IMAGE_TIMESTAMP:  556edb83

DEFAULT_BUCKET_ID:  VISTA_DRIVER_FAULT

BUGCHECK_STR:  0xF2

CURRENT_IRQL:  0

ANALYSIS_VERSION: 6.3.9600.16384 (debuggers(dbg).130821-1623) amd64fre

LAST_CONTROL_TRANSFER:  from 81591cb7 to 81459ba2

STACK_TEXT:  
WARNING: Stack unwind information not available. Following frames may be wrong.
815b4c1c 81591cb7 000000f2 81460e58 86114000 nt!KeBugCheckEx+0x1e
815b4c78 8d3b48da 00000000 00000000 86111958 nt!KeSynchronizeExecution+0x5f7
815b4ce4 81caaeaa 864fc0e8 815b9a90 815b4d50 netkvm+0x28da
815b4cf4 8154c449 00000000 00000000 815bb940 hal!HalAcquireDisplayOwnership+0x91e
815b4d50 815925af 00000000 0000000e 00000000 nt!PoSetPowerState+0x95d
815b4d54 00000000 0000000e 00000000 00000000 nt!KiDispatchInterrupt+0x63f


STACK_COMMAND:  kb

FOLLOWUP_IP: 
netkvm+28da
8d3b48da 84c0            test    al,al

SYMBOL_STACK_INDEX:  2

SYMBOL_NAME:  netkvm+28da

FOLLOWUP_NAME:  MachineOwner

IMAGE_NAME:  netkvm.sys

BUCKET_ID:  WRONG_SYMBOLS

FAILURE_BUCKET_ID:  WRONG_SYMBOLS

ANALYSIS_SOURCE:  KM

FAILURE_ID_HASH_STRING:  km:wrong_symbols

FAILURE_ID_HASH:  {70b057e8-2462-896f-28e7-ac72d4d365f8}

Followup: MachineOwner
---------
Comment 6 Yu Wang 2016-09-20 02:22:04 EDT
Hi

Re-test longevity test on rhel7, there is no bsod occurred when running netperf.exe for 2 days. So this bug has been fixed

virtio-win-1.9.0-2.el7.noarch
kernel-3.10.0-507.el7.x86_64
qemu-kvm-rhev-2.6.0-25.el7.x86_64
seabios-bin-1.9.1-5.el7.noarch


Thanks
Yu Wang
Comment 8 errata-xmlrpc 2016-11-04 04:52:05 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2609.html

Note You need to log in before you can comment on or make changes to this bug.