Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1857045

Summary: [virtio-win][netkvm] Windows guest BSOD when install netkvm driver after enable "0x009209bb" flag in verifer on virtio-win-prewhql-186
Product: Red Hat Enterprise Linux 8 Reporter: menli <menli>
Component: virtio-winAssignee: ybendito
virtio-win sub component: virtio-win-prewhql QA Contact: xiagao
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: juzhang, lijin, vrozenfe, xiagao, ybendito, yvugenfi
Version: 8.2Keywords: Regression
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-04 04:17:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description menli@redhat.com 2020-07-15 02:08:23 UTC
Description of problem:

Windows guest BSOD when install netkvm driver after enable "0x009209bb" flag in verifer on virtio-win-prewhql-186 

Version-Release number of selected component (if applicable):
qemu-kvm-4.2.0-19.module+el8.2.0+6296+6b821950.x86_64
kernel-4.18.0-193.el8.x86_64
seabios-1.13.0-1.module+el8.2.0+5520+4e5817f3.x86_64
virtio-win-prewhql-0.1-186.iso

guest: Win2016


How reproducible:
100%

Steps to Reproduce:
1.Start guest with qemu cmds:
MALLOC_PERTURB_=1  /usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1'  \
    -sandbox on  \
    -machine q35 \
    -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \
    -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0  \
    -nodefaults \
    -device VGA,bus=pcie.0,addr=0x2 \
    -m 14336  \
    -smp 24,maxcpus=24,cores=12,threads=1,dies=1,sockets=2  \
    -cpu 'Skylake-Server',hv_stimer,hv_synic,hv_vpindex,hv_relaxed,hv_spinlocks=0xfff,hv_vapic,hv_time,hv_frequencies,hv_runtime,hv_tlbflush,hv_reenlightenment,hv_stimer_direct,hv_ipi,+kvm_pv_unhalt \
    -device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \
    -device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
    -blockdev node-name=file_image1,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/win2016-64-virtio.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \
    -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,write-cache=on,bus=pcie-root-port-2,addr=0x0 \
    -device rtl8139,mac=9a:da:c9:87:9e:de,id=id66n7cc,netdev=idv9hlTZ,bus=pcie-pci-bridge-0,addr=0x1  \
    -netdev tap,id=idv9hlTZ \
    -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \
    -device virtio-net-pci,mac=9a:da:c9:87:9e:df,id=id7ANcTT,netdev=idmcFCC2,bus=pcie-root-port-3,addr=0x0  \
    -netdev tap,id=idmcFCC2 \
    -blockdev node-name=file_cd1,driver=file,read-only=on,aio=threads,filename=/home/kvm_autotest_root/iso/windows/winutils.iso,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_cd1,driver=raw,read-only=on,cache.direct=on,cache.no-flush=off,file=file_cd1 \
    -device ide-cd,id=cd1,drive=drive_cd1,bootindex=1,write-cache=on,bus=ide.0,unit=0 \
    -blockdev node-name=file_virtio,driver=file,read-only=on,aio=threads,filename=/home/kvm_autotest_root/iso/windows/virtio-win-prewhql-0.1-186.iso,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_virtio,driver=raw,read-only=on,cache.direct=on,cache.no-flush=off,file=file_virtio \
    -device ide-cd,id=virtio,drive=drive_virtio,bootindex=2,write-cache=on,bus=ide.1,unit=0  \
    -vnc :10  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot menu=off,order=cdn,once=c,strict=off \
    -enable-kvm \
    -device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x3,chassis=5 \
    -qmp tcp:0:1231,server,nowait \
    -monitor stdio \


2.Execute command: verifier /flags 0x009209BB /driver netkvm.sys

3. Reboot the guest: shutdown -r -t 0

4.Sending command: D:\devcon\win7_amd64\devcon.exe updateni E:\NetKVM\2k16\amd64\netkvm.inf "PCI\VEN_1AF4&DEV_1041"



Actual results:
windows guest hit BSOD.

Expected results:
Netkvm driver can be installed successfully.


Additional info:
1. Tried with virtio-win-prewhql-178, not reproduced this issue, so it should be a regression.
2. Tried with virtio-win-prewhql-185, not reproduced this issue.
3. only this issue on win2016 ,not hit on win2019

dump file:
http://fileshare.englab.nay.redhat.com/pub/section2/coredump/bz1857045/

Comment 1 xiagao 2020-07-15 03:59:28 UTC
@menli,
Confirm with you the steps.
1) install netkvm driver(186 version) during win2016 guest os installation
2) verifier netkvm driver
3) install the driver again, then hit BSOD, right?

Comment 2 menli@redhat.com 2020-07-15 04:17:37 UTC
(In reply to xiagao from comment #1)
> @menli,
> Confirm with you the steps.
> 1) install netkvm driver(186 version) during win2016 guest os installation
> 2) verifier netkvm driver
> 3) install the driver again, then hit BSOD, right?

xiagao, yes.the steps are right.

Comment 3 Yvugenfi@redhat.com 2020-07-15 08:29:20 UTC
BTW: Please pay attention when running driver verifier tests that for network and storage driver you need to add an appropriate MS port driver to the verifier:
https://www.osr.com/nt-insider/2016-issue1/analysts-perspective-my-driver-passes-driver-verifier/

Comment 4 Yvugenfi@redhat.com 2020-07-15 08:30:32 UTC
And for the rest of the drivers add wdfxxxxx.sys (were xxxxx is the appropriate version found in the specific OS).

Comment 5 Yvugenfi@redhat.com 2020-07-15 09:44:12 UTC
In any case, please don't test NetKVM in build 186. The 186 version has a bug that was taken from upstream and now is fixed.

Comment 6 lijin 2020-07-15 10:01:31 UTC
(In reply to Yan Vugenfirer from comment #5)
> In any case, please don't test NetKVM in build 186. The 186 version has a
> bug that was taken from upstream and now is fixed.

Yes, the acceptance testing is triggered automatically in CI once new prewhql build come out, we need a bug to track the failure.
We will verify with build 187 if this issue is gone.

Comment 7 Yvugenfi@redhat.com 2020-07-15 10:48:21 UTC
According to the crash dump protocol driver object is not cleaned-up (not related to the control queue bug in build 186).


nt!_VF_TARGET_DRIVER 0xffff94079f8c0b90: netkvm.sys (Loaded)

    Pool Allocation Statistics: ( NonPagedPool / PagedPool )

      Current Pool Allocations: ( 0x00000001 / 0x00000000 )
      Current Pool Bytes:       ( 0x00000070 / 0x00000000 )
      Peak Pool Allocations:    ( 0x00000001 / 0x00000000 )
      Peak Pool Bytes:          ( 0x00000070 / 0x00000000 )
      Contiguous Memory Bytes:       0x00000000
      Peak Contiguous Memory Bytes:  0x00000000

    Pool Allocations:

      Address             Length      Tag   Caller Address    
      ------------------  ----------  ----  ------------------
      0xffff91843fe2ef90  0x00000070  PROT  0xfffff805d39ba882  netkvm!ParaNdis_ProtocolInitialize+0x2a

    Contiguous allocations are not displayed with public symbols.

Comment 8 Yvugenfi@redhat.com 2020-07-15 10:51:48 UTC
According to the crash dump protocol driver object is not cleaned-up (not related to the control queue bug in build 186).


nt!_VF_TARGET_DRIVER 0xffff94079f8c0b90: netkvm.sys (Loaded)

    Pool Allocation Statistics: ( NonPagedPool / PagedPool )

      Current Pool Allocations: ( 0x00000001 / 0x00000000 )
      Current Pool Bytes:       ( 0x00000070 / 0x00000000 )
      Peak Pool Allocations:    ( 0x00000001 / 0x00000000 )
      Peak Pool Bytes:          ( 0x00000070 / 0x00000000 )
      Contiguous Memory Bytes:       0x00000000
      Peak Contiguous Memory Bytes:  0x00000000

    Pool Allocations:

      Address             Length      Tag   Caller Address    
      ------------------  ----------  ----  ------------------
      0xffff91843fe2ef90  0x00000070  PROT  0xfffff805d39ba882  netkvm!ParaNdis_ProtocolInitialize+0x2a

    Contiguous allocations are not displayed with public symbols.

Comment 10 xiagao 2020-08-10 02:02:58 UTC
Can reproduce this bug with coment0's steps on 186 version.

Verify this bug on virtio-win-prewhql-0.1-188.iso.

So it is fixed, change bug status to verified.

Comment 15 errata-xmlrpc 2020-11-04 04:17:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virtio-win bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:4840