1433830 – [virtio-win][whql][netkvm] guests bsod(d1) when running job "NDISTest 6.5 - [2 Machine] - OffloadLSO"

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1433830 - [virtio-win][whql][netkvm] guests bsod(d1) when running job "NDISTest 6.5 - [2 Machine] - OffloadLSO"

Summary: [virtio-win][whql][netkvm] guests bsod(d1) when running job "NDISTest 6.5 - [...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	virtio-win
Sub Component:
Version:	7.4
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	high
Target Milestone:	rc
Target Release:	---
Assignee:	ybendito
QA Contact:	Virtualization Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2017-03-20 03:39 UTC by Yu Wang
Modified:	2017-08-01 12:58 UTC (History)
CC List:	5 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2017-08-01 12:58:08 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2017:2341	0	normal	SHIPPED_LIVE	virtio-win bug fix and enhancement update	2017-08-01 16:52:38 UTC

Description Yu Wang 2017-03-20 03:39:28 UTC

Description of problem:
guests bsod(d1) when running job "NDISTest 6.5 - [2 Machine] - OffloadLSO"

Version-Release number of selected component (if applicable):
virtio-win-prewhql-134

How reproducible:
100%

Steps to Reproduce:
1. boot guest with net device
 /usr/libexec/qemu-kvm -name 134NIC201264CFW -enable-kvm -m 6G -smp 8 -uuid 36393bc9-dab8-4b2f-958c-24f5e2db9abc -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/tmp/133NIC201264CFW,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -boot order=cd,menu=on -device piix3-usb-uhci,id=usb -drive file=134NIC201264CFW,if=none,id=drive-ide0-0-0,format=raw,serial=mike_cao,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=en_windows_server_2012_x64_dvd_915478.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=134NIC201264CFW.vfd,if=floppy,id=drive-fdc0-0-0,format=raw,cache=none -netdev tap,script=/etc/qemu-ifup,downscript=no,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=00:52:15:36:f1:fa -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=isa_serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -M pc -netdev tap,script=/etc/qemu-ifup-private,downscript=no,id=hostnet1,vhost=on,queues=8 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=00:52:64:18:dd:00,bus=pci.0,mq=on,vectors=18

2. submit job
3.

Actual results:
BSOD

Expected results:
Pass without bsod


Additional info:

*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck D1, {3c, 2, 0, 91996b91}

*** ERROR: Module load completed but symbols could not be loaded for netkvm.sys
*** ERROR: Module load completed but symbols could not be loaded for NDProt630.sys
Probably caused by : netkvm.sys ( netkvm+9b91 )

Followup: MachineOwner
---------

1: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high.  This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000003c, memory referenced
Arg2: 00000002, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write operation
Arg4: 91996b91, address which referenced memory

Debugging Details:
------------------


OVERLAPPED_MODULE: Address regions for 'NDProt630' and 'NDProt630.sys' overlap

READ_ADDRESS:  0000003c 

CURRENT_IRQL:  2

FAULTING_IP: 
netkvm+9b91
91996b91 8b4a3c          mov     ecx,dword ptr [edx+3Ch]

DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

BUGCHECK_STR:  AV

PROCESS_NAME:  System

ANALYSIS_VERSION: 6.3.9600.16384 (debuggers(dbg).130821-1623) amd64fre

TRAP_FRAME:  91c55528 -- (.trap 0xffffffff91c55528)
ErrCode = 00000000
eax=ab69e2c8 ebx=add9af80 ecx=ae4ccf38 edx=00000000 esi=add9af80 edi=ae4ccf38
eip=91996b91 esp=91c5559c ebp=91c5559c iopl=0         nv up ei pl zr na pe nc
cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00010246
netkvm+0x9b91:
91996b91 8b4a3c          mov     ecx,dword ptr [edx+3Ch] ds:0023:0000003c=????????
Resetting default scope

LAST_CONTROL_TRANSFER:  from 81530ca3 to 8151c5f0

STACK_TEXT:  
91c55480 81530ca3 0000000a 0000003c 00000002 nt!KiBugCheck2
91c55480 91996b91 0000000a 0000003c 00000002 nt!KiTrap0E+0x1cf
WARNING: Stack unwind information not available. Following frames may be wrong.
91c5559c 91994629 ab69e2c8 00000000 add9af80 netkvm+0x9b91
91c555b8 91994d28 ac1facf8 ae4ccf38 91995491 netkvm+0x7629
91c555e0 9199920d 014ccf38 9e4dfcd8 88fa6618 netkvm+0x7d28
91c555fc 82a21376 ab69e2c8 ae4ccf38 00000000 netkvm+0xc20d
91c55620 804a40ff ab69e2c8 ae4ccf38 00000000 VerifierExt!XdvNdisMiniportSendNetBufferListsHandler_wrapper+0xb6
91c55684 804ad606 9b4920e8 004ccf38 00000000 ndis!ndisMSendNBLToMiniport+0xffff6e4b
91c556c0 804a98b6 00000000 9b4920e8 9b4920e8 ndis!ndisInvokeNextSendHandler+0x22
91c55700 858517e6 af3a2df0 ae4ccf38 00000000 ndis!NdisFSendNetBufferLists+0x27d
91c55798 804ad67c 947ee898 ae4ccf38 00000000 wfplwfs!LwfLowerSendNetBufferLists+0x12d
91c557e4 804ad5db 858516be 947ee898 ae4ccf38 ndis!ndisCallSendHandler+0x20
91c55804 814a97a5 91c558cc 43f0512c af3a2df0 ndis!ndisDataPathExpandStackCallback+0x24
91c55894 814a9405 02001000 00000000 00000000 nt!KeExpandKernelStackAndCalloutInternal+0x385
91c558ac 804ad56d 804ad5b7 91c558cc 00001000 nt!KeExpandKernelStackAndCalloutEx+0x1f
91c558c4 804ad5ac 91c55948 947ee898 858516be ndis!ndisExpandStack+0x11
91c558ec 804ad644 00000000 00000000 947ee898 ndis!ndisExpandDataPathStack+0x39
91c55930 804a98b6 00000000 af3a2df0 947ee898 ndis!ndisInvokeNextSendHandler+0x60
91c55970 8b479179 af792df0 ae4ccf38 00000000 ndis!NdisFSendNetBufferLists+0x27d
91c55a28 804ad67c ac632f70 ae4ccf38 00000000 pacer!PcFilterSendNetBufferLists+0x4a
91c55a74 804ad5db 8b47912f ac632f70 ae4ccf38 ndis!ndisCallSendHandler+0x20
91c55a94 814a97a5 91c55b5c 43f0529c ae4ccf38 ndis!ndisDataPathExpandStackCallback+0x24
91c55b24 814a9405 00001000 00000000 00000000 nt!KeExpandKernelStackAndCalloutInternal+0x385
91c55b3c 804ad56d 804ad5b7 91c55b5c 00001000 nt!KeExpandKernelStackAndCalloutEx+0x1f
91c55b54 804ad5ac 00000000 ac632f70 8b47912f ndis!ndisExpandStack+0x11
91c55b7c 804a843d 00000000 00000000 ac632f70 ndis!ndisExpandDataPathStack+0x39
91c55c64 91838240 ac090de0 ae4ccf38 00000000 ndis!NdisSendNetBufferLists+0xc7b
91c55c8c 918acca4 ae4ccf38 00000000 00000000 NDProt630+0x38240
91c55cc0 918abe72 ae4ccf38 00000000 9e4e2940 NDProt630+0xacca4
91c55d50 9190b260 00000000 818d4e61 00000000 NDProt630+0xabe72
91c55d70 814b677a 86928044 43f05408 00000000 NDProt630+0x10b260
91c55db0 81531fe1 9190b220 86928044 00000000 nt!PspSystemThreadStartup+0x58
91c55dc8 81463726 91c55de0 81464387 847d6c28 nt!KiThreadStartup+0x15
91c55dd0 81464387 847d6c28 91c55e04 91c56000 nt!PsLeavePriorityRegion+0x12
91c55ddc 91c56000 91c53000 00000000 00000000 nt!KeReleaseInStackQueuedSpinLock+0x37
91c55e04 00000000 00000000 00000000 00000000 0x91c56000


STACK_COMMAND:  kb

FOLLOWUP_IP: 
netkvm+9b91
91996b91 8b4a3c          mov     ecx,dword ptr [edx+3Ch]

SYMBOL_STACK_INDEX:  2

SYMBOL_NAME:  netkvm+9b91

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: netkvm

IMAGE_NAME:  netkvm.sys

DEBUG_FLR_IMAGE_TIMESTAMP:  58c7e623

FAILURE_BUCKET_ID:  AV_VRF_netkvm+9b91

BUCKET_ID:  AV_VRF_netkvm+9b91

ANALYSIS_SOURCE:  KM

FAILURE_ID_HASH_STRING:  km:av_vrf_netkvm+9b91

FAILURE_ID_HASH:  {5851291b-827c-d914-a504-d0e388834932}

Followup: MachineOwner
---------

1: kd>

Comment 4 xiagao 2017-03-21 03:33:37 UTC

win2008r2,win7-32/64 and win8-32/64 hit the same issue.

Comment 5 xiagao 2017-03-21 07:09:19 UTC

win2008-32/64 hit the same issue in job "Ethernet - NDISTest 6.5"

Comment 7 xiagao 2017-03-22 06:43:53 UTC

2008-r2 and win7-64 hit same issue in job "DF - Concurrent Hardware And Operating System (CHAOS) Test (Certification)"

Comment 8 ybendito 2017-03-23 08:02:34 UTC

Fix prepared and merged to upstream

https://github.com/virtio-win/kvm-guest-drivers-windows/pull/117
https://github.com/virtio-win/kvm-guest-drivers-windows/commit/02d8d18e9de92defe5b6e18823eedb30797e4936

Comment 9 lijin 2017-05-26 06:03:57 UTC

Job pass with build 137.
Change status to verifed

Comment 12 errata-xmlrpc 2017-08-01 12:58:08 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2341

Note You need to log in before you can comment on or make changes to this bug.